Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1DAAD485F for ; Tue, 17 May 2011 12:50:49 +0000 (UTC) Received: (qmail 68843 invoked by uid 500); 17 May 2011 12:50:47 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 68817 invoked by uid 500); 17 May 2011 12:50:47 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 68809 invoked by uid 99); 17 May 2011 12:50:47 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 May 2011 12:50:47 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of sylvain@datastax.com designates 209.85.213.44 as permitted sender) Received: from [209.85.213.44] (HELO mail-yw0-f44.google.com) (209.85.213.44) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 May 2011 12:50:40 +0000 Received: by ywp31 with SMTP id 31so143541ywp.31 for ; Tue, 17 May 2011 05:50:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.236.177.65 with SMTP id c41mr507448yhm.463.1305636619606; Tue, 17 May 2011 05:50:19 -0700 (PDT) Received: by 10.236.103.165 with HTTP; Tue, 17 May 2011 05:50:19 -0700 (PDT) X-Originating-IP: [88.183.33.171] In-Reply-To: <15FEFA0410B1AA4F9775D72F0279A368C1AB4B@EU-EXCH02.nuance.com> References: <15FEFA0410B1AA4F9775D72F0279A368C1A941@EU-EXCH02.nuance.com> <3A0D4D0E-843A-4DC9-A6F1-1884CF12E458@thelastpickle.com> <15FEFA0410B1AA4F9775D72F0279A368C1AB1B@EU-EXCH02.nuance.com> <15FEFA0410B1AA4F9775D72F0279A368C1AB4B@EU-EXCH02.nuance.com> Date: Tue, 17 May 2011 14:50:19 +0200 Message-ID: Subject: Re: AssertionError From: Sylvain Lebresne To: user@cassandra.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Tue, May 17, 2011 at 1:46 PM, Desimpel, Ignace wrote: > Ok, I will do that (next test will be done on some linux boxes being inst= alled now, but at this time I need to gone with the current windows setup). > Question : Can I use the 0.7.4 data files as is? Do I need to backup the = datafiles in order to be able to get back to the 0.7.4 version if needed? Yes you can use 0.7.4 data files as is. And I can't think of a reason why you should have problem getting back to 0.7.4 if needed, though snapshotting before cannot hurt. > > Ignace > > -----Original Message----- > From: Sylvain Lebresne [mailto:sylvain@datastax.com] > Sent: Tuesday, May 17, 2011 1:16 PM > To: user@cassandra.apache.org > Subject: Re: AssertionError > > First thing to do would be to update to 0.7.5. > > The assertionError you're running into is a assertion where we check if a= skipBytes did skip all the bytes we had ask him to. As it turns out, the s= pec for skipBytes authorize it to not skip all the bytes asked even with no= good reason. I'm pretty sure on a linux box skipBytes on a file will alway= s read the number of asked bytes unless it reaches EOF, but I see you're ru= nning windows, so who knows what can happen. > > Anyway, long story short, it's a "bug" in 0.7.4 that has been fixed in 0.= 7.5. If you still run into this in 0.7.5 at least we'll know it's something= else (and we will have a more helpful error message). > > -- > Sylvain > > On Tue, May 17, 2011 at 12:41 PM, Desimpel, Ignace wrote: >> I use a custom comparator class. So I think there is a high chance >> that I do something wrong there. I was thinking that the stack trace >> could give a clue and help me on the way, maybe because some already got= the same error. >> >> >> >> Anyway, here is some more information you requested. >> >> >> >> Yaml definition : >> >> name: ForwardStringValues >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0 column_type: Super >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0 compare_with: >> be.landc.services.search.server.db.cassandra.node.ForwardCFStringValue >> s >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0 compare_subcolumns_with: BytesType >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0 keys_cached: 100000 >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0 rows_cached: 0 >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0 comment: Stores the values of functions retu= rning string >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0 memtable_throughput_in_mb: 64 >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0 memtable_operations_in_millions: 15 >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0 min_compaction_threshold: 2 >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0 max_compaction_threshold: 5 >> >> >> >> Column Family: ForwardStringValues >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 SSTable count: 8 >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Space used (live): 1313117= 76690 >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Space used (total): 131311= 776690 >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Memtable Columns Count: 0 >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Memtable Data Size: 0 >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Memtable Switch Count: 0 >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Read Count: 1 >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Read Latency: 404.890 ms. >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Write Count: 0 >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Write Latency: NaN ms. >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Pending Tasks: 0 >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Key cache capacity: 100000 >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Key cache size: 8 >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Key cache hit rate: 1.0 >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Row cache: disabled >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Compacted row minimum size= : 150 >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Compacted row maximum size= : 7152383774 >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Compacted row mean size: 3= 064535 >> >> >> >> No secondary indexes. >> >> Total database disk size 823 Gb >> >> disk_access_mode: auto on 64 bit windows os >> >> partitioner: org.apache.cassandra.dht.ByteOrderedPartitioner >> >> Data was stored over a period of 5 days. >> >> Cassandra 0.7.4 was running as an embedded server. >> >> Batch insert, using the StorageProxy.mutate. >> >> No errors were logged during the batch insert period. >> >> The row key is a string representation of a positive integer value. >> >> The same row key is used during many different mutate calls, but all >> super column names are different for each call. >> >> The column name of the super class stored =A0is composed of the 32 bytes >> and the bytes of 2 integer (positive and negative) values and the >> bytes (UTF8) of the string value :[32 bytes][4 int bytes][4 int >> bytes][string bytes] >> >> The custom comparator class ...ForwardCFStringValues sorts the names >> by first sorting the string , then the 32 bytes, and then the two >> integer values >> >> For each column name two subcolumns are inserted with fixed name and >> some small binary value (about 40 bytes) >> >> >> >> The query : >> >> Get_slice using thrift. >> >> Params : >> >> =A0=A0Row key : the string representation of the positive integer String= '1788' >> thus hex values 31 37 38 38 >> >> =A0 ColumnParent : the column family ForwardStringValues >> >> =A0 SlicePredicate : SlicePredicate(slice_range:SliceRange(start:00 00 >> 00 00 >> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 >> 00 00 >> 00 00 00 FF FF FF FF FF FF FF FF 55 52 49 4E 41 52 59 20 54 52 41 43 >> 54 20 >> 49 4E 46 45 43 54 49 4F 4E, finish:7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F >> 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F FF FF FF >> 7F FF FF FF 7F 55 52 49 4E 41 52 59 20 54 52 41 43 54 20 49 4E 46 45 >> 43 54 49 4F 4E, reversed:false, count:10000)) >> >> >> >> This SlicePredicate is supposed to fetch all the columns with the >> string '55 >> 52 49 4E 41 52 59 20 54 52 41 43 54 20 49 4E 46 45 43 54 49 4F 4E' >> regardless of the other bytes in the column name. So the start and >> finish have the same string bytes. The rest of the bytes for the start >> values are set to the lowest possible value (32 zero bytes and the >> bytes FFFFFFFF representing the integer value -1) , the finish is set >> the highest possible value (32 bytes with value 7F, ...) >> >> >> >> I tested the same code but with a small data set and all seemed to be OK= . >> Even on the same database I get back results without exception if I >> use different String values. I'm almost sure that there should be >> columns with that string. If the string is not present I don't get the e= rror. >> >> >> >> >> >> >> >> >> >> From: Aaron Morton [mailto:aaron@thelastpickle.com] >> Sent: Monday, May 16, 2011 11:33 PM >> To: user@cassandra.apache.org >> Subject: Re: AssertionError >> >> >> >> The code is trying to follow the column index for a row in an sstable, >> but it cannot skip as many bytes as it would like to to get to the colum= n. >> Helpfully the help says running out of bytes is only one of the >> reasons why this could happen:) >> >> >> >> Can you provide some more information about the query and the data, >> and also the upgrade history for your cluster. >> >> >> >> Thanks >> >> Aaron >> >> On 17/05/2011, at 3:07 AM, "Desimpel, Ignace" >> >> wrote: >> >> Environment : java 64 bit server, java client, thrift get_slice >> method, Cassandra 0.7.4, single node >> >> Depending on the data I pass for a query on a CF I get the following >> listed below. Any suggestions what could be wrong based on the stack tra= ce? >> >> >> >> java.lang.AssertionError >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at >> org.apache.cassandra.db.columniterator.IndexedSliceReader$IndexedBlock >> Fetcher.getNextBlock(IndexedSliceReader.java:176) >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at >> org.apache.cassandra.db.columniterator.IndexedSliceReader.computeNext( >> IndexedSliceReader.java:120) >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at >> org.apache.cassandra.db.columniterator.IndexedSliceReader.computeNext( >> IndexedSliceReader.java:48) >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at >> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIt >> erator.java:136) >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at >> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.ja >> va:131) >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at >> org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SS >> TableSliceIterator.java:108) >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at >> org.apache.commons.collections.iterators.CollatingIterator.set(Collati >> ngIterator.java:282) >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at >> org.apache.commons.collections.iterators.CollatingIterator.least(Colla >> tingIterator.java:325) >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at >> org.apache.commons.collections.iterators.CollatingIterator.next(Collat >> ingIterator.java:229) >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at >> org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterat >> or.java:68) >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at >> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIt >> erator.java:136) >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at >> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.ja >> va:131) >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at >> org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns( >> SliceQueryFilter.java:116) >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at >> org.apache.cassandra.db.filter.QueryFilter.collectCollatedColumns(Quer >> yFilter.java:130) >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at >> org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFam >> ilyStore.java:1368) >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at >> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamily >> Store.java:1245) >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at >> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamily >> Store.java:1173) >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at >> org.apache.cassandra.db.Table.getRow(Table.java:333) >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at >> org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadComma >> nd.java:63) >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at >> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThro >> w(StorageProxy.java:453) >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at >> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30 >> ) >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at >> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecu >> tor.java:886) >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor. >> java:908) >> >> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at java.lang.Thread.run(Th= read.java:662) >> >> >> >> Ignace Desimpel >