Return-Path: X-Original-To: apmail-cassandra-commits-archive@www.apache.org Delivered-To: apmail-cassandra-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C16C1988B for ; Thu, 8 Dec 2011 12:04:19 +0000 (UTC) Received: (qmail 50859 invoked by uid 500); 8 Dec 2011 12:04:19 -0000 Delivered-To: apmail-cassandra-commits-archive@cassandra.apache.org Received: (qmail 50830 invoked by uid 500); 8 Dec 2011 12:04:19 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 50818 invoked by uid 99); 8 Dec 2011 12:04:19 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 08 Dec 2011 12:04:19 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 08 Dec 2011 12:04:17 +0000 Received: from eris.apache.org (localhost [127.0.0.1]) by eris.apache.org (Postfix) with ESMTP id 9D0182388860 for ; Thu, 8 Dec 2011 12:03:56 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1211848 - in /cassandra/branches/cassandra-1.0: ./ src/java/org/apache/cassandra/io/sstable/ test/data/legacy-sstables/hb/ test/data/legacy-sstables/hb/Keyspace1/ test/unit/org/apache/cassandra/io/sstable/ Date: Thu, 08 Dec 2011 12:03:56 -0000 To: commits@cassandra.apache.org From: slebresne@apache.org X-Mailer: svnmailer-1.0.8-patched Message-Id: <20111208120356.9D0182388860@eris.apache.org> Author: slebresne Date: Thu Dec 8 12:03:55 2011 New Revision: 1211848 URL: http://svn.apache.org/viewvc?rev=1211848&view=rev Log: add back partitioner in sstable metadata patch by yukim; reviewed by slebresne for CASSANDRA-3540 Added: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/ cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/ cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1-hb-1-Data.db (with props) cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1-hb-1-Digest.sha1 cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1-hb-1-Filter.db (with props) cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1-hb-1-Index.db (with props) cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1-hb-1-Statistics.db (with props) cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1.626972746864617465-hb-1-Data.db (with props) cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1.626972746864617465-hb-1-Digest.sha1 cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1.626972746864617465-hb-1-Filter.db (with props) cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1.626972746864617465-hb-1-Index.db (with props) cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1.626972746864617465-hb-1-Statistics.db (with props) cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Standard1-hb-0-Data.db (with props) cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Standard1-hb-0-Digest.sha1 cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Standard1-hb-0-Filter.db (with props) cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Standard1-hb-0-Index.db (with props) cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Standard1-hb-0-Statistics.db (with props) Modified: cassandra/branches/cassandra-1.0/CHANGES.txt cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/io/sstable/SSTableMetadata.java cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/io/sstable/SSTableReader.java cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java cassandra/branches/cassandra-1.0/test/unit/org/apache/cassandra/io/sstable/SSTableMetadataSerializerTest.java cassandra/branches/cassandra-1.0/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java Modified: cassandra/branches/cassandra-1.0/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/CHANGES.txt?rev=1211848&r1=1211847&r2=1211848&view=diff ============================================================================== --- cassandra/branches/cassandra-1.0/CHANGES.txt (original) +++ cassandra/branches/cassandra-1.0/CHANGES.txt Thu Dec 8 12:03:55 2011 @@ -17,6 +17,7 @@ (CASSANDRA-3558) * fix missing response during range slice repair (CASSANDRA-3551) * 'describe ring' moved from CLI to nodetool and available through JMX (CASSANDRA-3220) + * add back partitioner to sstable metadata (CASSANDRA-3540) Merged from 0.8: * use cannonical host for local node in nodetool info (CASSANDRA-3556) * remove nonlocal DC write optimization since it only worked with Modified: cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/io/sstable/SSTableMetadata.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/io/sstable/SSTableMetadata.java?rev=1211848&r1=1211847&r2=1211848&view=diff ============================================================================== --- cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/io/sstable/SSTableMetadata.java (original) +++ cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/io/sstable/SSTableMetadata.java Thu Dec 8 12:03:55 2011 @@ -29,7 +29,6 @@ import java.io.IOException; import org.slf4j.Logger; import org.slf4j.LoggerFactory; -import org.apache.cassandra.config.DatabaseDescriptor; import org.apache.cassandra.db.commitlog.ReplayPosition; import org.apache.cassandra.io.util.FileUtils; import org.apache.cassandra.utils.EstimatedHistogram; @@ -67,7 +66,7 @@ public class SSTableMetadata ReplayPosition.NONE, Long.MIN_VALUE, Double.MIN_VALUE, - DatabaseDescriptor.getPartitioner().getClass().getCanonicalName()); + null); } private SSTableMetadata(EstimatedHistogram rowSizes, EstimatedHistogram columnCounts, ReplayPosition replayPosition, long maxTimestamp, double cr, String partitioner) @@ -134,14 +133,14 @@ public class SSTableMetadata maxTimestamp = Math.max(maxTimestamp, potentialMax); } - public SSTableMetadata finalizeMetadata() + public SSTableMetadata finalizeMetadata(String partitioner) { return new SSTableMetadata(estimatedRowSize, estimatedColumnCount, replayPosition, maxTimestamp, compressionRatio, - DatabaseDescriptor.getPartitioner().getClass().getCanonicalName()); + partitioner); } public Collector estimatedRowSize(EstimatedHistogram estimatedRowSize) @@ -169,6 +168,8 @@ public class SSTableMetadata public void serialize(SSTableMetadata sstableStats, DataOutput dos) throws IOException { + assert sstableStats.partitioner != null; + EstimatedHistogram.serializer.serialize(sstableStats.estimatedRowSize, dos); EstimatedHistogram.serializer.serialize(sstableStats.estimatedColumnCount, dos); ReplayPosition.serializer.serialize(sstableStats.replayPosition, dos); @@ -209,9 +210,7 @@ public class SSTableMetadata double compressionRatio = desc.hasCompressionRatio ? dis.readDouble() : Double.MIN_VALUE; - String partitioner = desc.hasPartitioner - ? dis.readUTF() - : DatabaseDescriptor.getPartitioner().getClass().getCanonicalName(); + String partitioner = desc.hasPartitioner ? dis.readUTF() : null; return new SSTableMetadata(rowSizes, columnCounts, replayPosition, maxTimestamp, compressionRatio, partitioner); } } Modified: cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/io/sstable/SSTableReader.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/io/sstable/SSTableReader.java?rev=1211848&r1=1211847&r2=1211848&view=diff ============================================================================== --- cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/io/sstable/SSTableReader.java (original) +++ cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/io/sstable/SSTableReader.java Thu Dec 8 12:03:55 2011 @@ -137,9 +137,11 @@ public class SSTableReader extends SSTab ? SSTableMetadata.serializer.deserialize(descriptor) : SSTableMetadata.createDefaultInstance(); - // check if sstable is created using same partitioner as this node + // Check if sstable is created using same partitioner. + // Partitioner can be null, which indicates older version of sstable or no stats available. + // In that case, we skip the check. String partitionerName = partitioner.getClass().getCanonicalName(); - if (!partitionerName.equals(sstableMetadata.partitioner)) + if (sstableMetadata.partitioner != null && !partitionerName.equals(sstableMetadata.partitioner)) throw new RuntimeException(String.format("Cannot open %s because partitioner does not match %s", descriptor, partitionerName)); Modified: cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java?rev=1211848&r1=1211847&r2=1211848&view=diff ============================================================================== --- cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java (original) +++ cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/io/sstable/SSTableWriter.java Thu Dec 8 12:03:55 2011 @@ -311,7 +311,7 @@ public class SSTableWriter extends SSTab dataFile.close(); // write sstable statistics - SSTableMetadata sstableMetadata = sstableMetadataCollector.finalizeMetadata(); + SSTableMetadata sstableMetadata = sstableMetadataCollector.finalizeMetadata(partitioner.getClass().getCanonicalName()); writeMetadata(descriptor, sstableMetadata); maybeWriteDigest(); Added: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1-hb-1-Data.db URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1-hb-1-Data.db?rev=1211848&view=auto ============================================================================== Binary file - no diff available. Propchange: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1-hb-1-Data.db ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1-hb-1-Digest.sha1 URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1-hb-1-Digest.sha1?rev=1211848&view=auto ============================================================================== --- cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1-hb-1-Digest.sha1 (added) +++ cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1-hb-1-Digest.sha1 Thu Dec 8 12:03:55 2011 @@ -0,0 +1 @@ +095dd05150d499846782cdf1c77544048477cc9b Indexed1-hb-1-Data.db \ No newline at end of file Added: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1-hb-1-Filter.db URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1-hb-1-Filter.db?rev=1211848&view=auto ============================================================================== Binary file - no diff available. Propchange: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1-hb-1-Filter.db ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1-hb-1-Index.db URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1-hb-1-Index.db?rev=1211848&view=auto ============================================================================== Binary file - no diff available. Propchange: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1-hb-1-Index.db ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1-hb-1-Statistics.db URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1-hb-1-Statistics.db?rev=1211848&view=auto ============================================================================== Binary file - no diff available. Propchange: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1-hb-1-Statistics.db ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1.626972746864617465-hb-1-Data.db URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1.626972746864617465-hb-1-Data.db?rev=1211848&view=auto ============================================================================== Binary file - no diff available. Propchange: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1.626972746864617465-hb-1-Data.db ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1.626972746864617465-hb-1-Digest.sha1 URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1.626972746864617465-hb-1-Digest.sha1?rev=1211848&view=auto ============================================================================== --- cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1.626972746864617465-hb-1-Digest.sha1 (added) +++ cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1.626972746864617465-hb-1-Digest.sha1 Thu Dec 8 12:03:55 2011 @@ -0,0 +1 @@ +a6a32d25ca3cdf76ac5a1a6bffa78936463f9648 Indexed1.626972746864617465-hb-1-Data.db \ No newline at end of file Added: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1.626972746864617465-hb-1-Filter.db URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1.626972746864617465-hb-1-Filter.db?rev=1211848&view=auto ============================================================================== Binary file - no diff available. Propchange: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1.626972746864617465-hb-1-Filter.db ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1.626972746864617465-hb-1-Index.db URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1.626972746864617465-hb-1-Index.db?rev=1211848&view=auto ============================================================================== Binary file - no diff available. Propchange: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1.626972746864617465-hb-1-Index.db ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1.626972746864617465-hb-1-Statistics.db URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1.626972746864617465-hb-1-Statistics.db?rev=1211848&view=auto ============================================================================== Binary file - no diff available. Propchange: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Indexed1.626972746864617465-hb-1-Statistics.db ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Standard1-hb-0-Data.db URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Standard1-hb-0-Data.db?rev=1211848&view=auto ============================================================================== Binary file - no diff available. Propchange: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Standard1-hb-0-Data.db ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Standard1-hb-0-Digest.sha1 URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Standard1-hb-0-Digest.sha1?rev=1211848&view=auto ============================================================================== --- cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Standard1-hb-0-Digest.sha1 (added) +++ cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Standard1-hb-0-Digest.sha1 Thu Dec 8 12:03:55 2011 @@ -0,0 +1 @@ +72295cc1b64e6ffdb67a39dec9996c4ff0363cce Standard1-hb-0-Data.db \ No newline at end of file Added: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Standard1-hb-0-Filter.db URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Standard1-hb-0-Filter.db?rev=1211848&view=auto ============================================================================== Binary file - no diff available. Propchange: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Standard1-hb-0-Filter.db ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Standard1-hb-0-Index.db URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Standard1-hb-0-Index.db?rev=1211848&view=auto ============================================================================== Binary file - no diff available. Propchange: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Standard1-hb-0-Index.db ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Standard1-hb-0-Statistics.db URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Standard1-hb-0-Statistics.db?rev=1211848&view=auto ============================================================================== Binary file - no diff available. Propchange: cassandra/branches/cassandra-1.0/test/data/legacy-sstables/hb/Keyspace1/Standard1-hb-0-Statistics.db ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Modified: cassandra/branches/cassandra-1.0/test/unit/org/apache/cassandra/io/sstable/SSTableMetadataSerializerTest.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/test/unit/org/apache/cassandra/io/sstable/SSTableMetadataSerializerTest.java?rev=1211848&r1=1211847&r2=1211848&view=diff ============================================================================== --- cassandra/branches/cassandra-1.0/test/unit/org/apache/cassandra/io/sstable/SSTableMetadataSerializerTest.java (original) +++ cassandra/branches/cassandra-1.0/test/unit/org/apache/cassandra/io/sstable/SSTableMetadataSerializerTest.java Thu Dec 8 12:03:55 2011 @@ -28,6 +28,7 @@ import java.io.IOException; import org.junit.Test; import org.apache.cassandra.db.commitlog.ReplayPosition; +import org.apache.cassandra.dht.RandomPartitioner; import org.apache.cassandra.utils.EstimatedHistogram; public class SSTableMetadataSerializerTest @@ -49,7 +50,7 @@ public class SSTableMetadataSerializerTe .estimatedColumnCount(columnCounts) .replayPosition(rp); collector.updateMaxTimestamp(maxTimestamp); - SSTableMetadata originalMetadata = collector.finalizeMetadata(); + SSTableMetadata originalMetadata = collector.finalizeMetadata(RandomPartitioner.class.getCanonicalName()); ByteArrayOutputStream byteOutput = new ByteArrayOutputStream(); DataOutputStream dos = new DataOutputStream(byteOutput); @@ -69,5 +70,6 @@ public class SSTableMetadataSerializerTe assert stats.replayPosition.equals(rp); assert stats.maxTimestamp == maxTimestamp; assert stats.maxTimestamp == originalMetadata.maxTimestamp; + assert RandomPartitioner.class.getCanonicalName().equals(stats.partitioner); } } Modified: cassandra/branches/cassandra-1.0/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java?rev=1211848&r1=1211847&r2=1211848&view=diff ============================================================================== --- cassandra/branches/cassandra-1.0/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java (original) +++ cassandra/branches/cassandra-1.0/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java Thu Dec 8 12:03:55 2011 @@ -21,6 +21,7 @@ package org.apache.cassandra.io.sstable; */ +import java.io.File; import java.io.IOException; import java.nio.ByteBuffer; import java.util.concurrent.ExecutionException; @@ -33,14 +34,21 @@ import org.junit.Test; import org.apache.cassandra.CleanupHelper; import org.apache.cassandra.config.DatabaseDescriptor; import org.apache.cassandra.db.*; +import org.apache.cassandra.db.columniterator.IdentityQueryFilter; import org.apache.cassandra.db.compaction.CompactionManager; import org.apache.cassandra.db.filter.QueryPath; +import org.apache.cassandra.dht.IPartitioner; import org.apache.cassandra.dht.Range; import org.apache.cassandra.dht.Token; import org.apache.cassandra.io.util.FileDataInput; +import org.apache.cassandra.io.util.FileUtils; import org.apache.cassandra.io.util.MmappedSegmentedFile; import org.apache.cassandra.service.StorageService; +import org.apache.cassandra.thrift.IndexClause; +import org.apache.cassandra.thrift.IndexExpression; +import org.apache.cassandra.thrift.IndexOperator; import org.apache.cassandra.utils.ByteBufferUtil; +import org.apache.cassandra.utils.CLibrary; import org.apache.cassandra.utils.Pair; import org.apache.cassandra.Util; @@ -188,6 +196,75 @@ public class SSTableReaderTest extends C assert p.right == p7; } + @Test + public void testPersistentStatisticsWithSecondaryIndex() throws IOException, ExecutionException, InterruptedException + { + // Create secondary index and flush to disk + Table table = Table.open("Keyspace1"); + ColumnFamilyStore store = table.getColumnFamilyStore("Indexed1"); + ByteBuffer key = ByteBufferUtil.bytes(String.valueOf("k1")); + RowMutation rm = new RowMutation("Keyspace1", key); + rm.add(new QueryPath("Indexed1", null, ByteBufferUtil.bytes("birthdate")), ByteBufferUtil.bytes(1L), System.currentTimeMillis()); + rm.apply(); + store.forceBlockingFlush(); + + // check if opening and querying works + assertIndexQueryWorks(store); + } + + @Test + public void testPersistentStatisticsFromOlderIndexedSSTable() throws IOException, ExecutionException, InterruptedException + { + // copy legacy indexed sstables + String root = System.getProperty("legacy-sstable-root"); + assert root != null; + File rootDir = new File(root + File.separator + "hb" + File.separator + "Keyspace1"); + assert rootDir.isDirectory(); + + String[] destDirs = DatabaseDescriptor.getAllDataFileLocationsForTable("Keyspace1"); + assert destDirs != null; + assert destDirs.length > 0; + + FileUtils.createDirectory(destDirs[0]); + for (File srcFile : rootDir.listFiles()) + { + if (!srcFile.getName().startsWith("Indexed1")) + continue; + File destFile = new File(destDirs[0] + File.separator + srcFile.getName()); + CLibrary.createHardLinkWithExec(srcFile, destFile); + + destFile = new File(destDirs[0] + File.separator + srcFile.getName()); + + assert destFile.exists() : destFile.getAbsoluteFile(); + } + ColumnFamilyStore store = Table.open("Keyspace1").getColumnFamilyStore("Indexed1"); + + // check if opening and querying works + assertIndexQueryWorks(store); + } + + private void assertIndexQueryWorks(ColumnFamilyStore indexedCFS) + { + assert "Indexed1".equals(indexedCFS.getColumnFamilyName()); + + // make sure all sstables including 2ary indexes load from disk + indexedCFS.clearUnsafe(); + for (ColumnFamilyStore indexCfs : indexedCFS.indexManager.getIndexesBackedByCfs()) + { + indexCfs.clearUnsafe(); + indexCfs.loadNewSSTables(); // v1.0.4 would fail here (see CASSANDRA-3540) + } + indexedCFS.loadNewSSTables(); + + // query using index to see if sstable for secondary index opens + IndexExpression expr = new IndexExpression(ByteBufferUtil.bytes("birthdate"), IndexOperator.EQ, ByteBufferUtil.bytes(1L)); + IndexClause clause = new IndexClause(Arrays.asList(expr), ByteBufferUtil.EMPTY_BYTE_BUFFER, 100); + IPartitioner p = StorageService.getPartitioner(); + Range range = new Range(p.getMinimumToken(), p.getMinimumToken()); + List rows = indexedCFS.search(clause, range, new IdentityQueryFilter()); + assert rows.size() == 1; + } + private List makeRanges(Token left, Token right) { return Arrays.asList(new Range[]{ new Range(left, right) });