incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Soumava Ghosh <soum...@cs.utexas.edu>
Subject Re: Cassandra 2.0 - AssertionError in ArrayBackedSortedColumns
Date Mon, 22 Jul 2013 08:42:38 GMT
Thanks for the confirmation!

It looks like the code is capable of handling the data in unsorted order
with assertions disabled; so for the time being I will disable the
assertion for my tests, until the fixes are available.

Thanks,
Soumava


On Mon, Jul 22, 2013 at 1:33 AM, Sylvain Lebresne <sylvain@datastax.com>wrote:

> Actually, your second stack trace is due to
> https://issues.apache.org/jira/browse/CASSANDRA-5788.
>
>
> On Mon, Jul 22, 2013 at 9:37 AM, Sylvain Lebresne <sylvain@datastax.com>wrote:
>
>>
>>  unless data was stored in incorrect order and I don't know if that is
>>> possible through the API..
>>>
>>
>> That's exactly what happens and that's why I say it's a bug.
>>
>>
>>>
>>> ii. I am also seeing some sporadic cases where a single column cas()
>>> update is hitting this same issue (the bug you mentioned) as multiple
>>> columns, do you see any reason for that happening?
>>>
>>
>> That would be slightly weirder, but frankly that's almost surely due to
>> https://issues.apache.org/jira/browse/CASSANDRA-5786 too.
>>
>> --
>> Sylvain
>>
>>
>>> Thank you,
>>> Soumava
>>>
>>>
>>> On Mon, Jul 22, 2013 at 12:11 AM, Sylvain Lebresne <sylvain@datastax.com
>>> > wrote:
>>>
>>>> This is a bug really:
>>>> https://issues.apache.org/jira/browse/CASSANDRA-5786.
>>>>
>>>> This should get fixed in the next beta of 2.0, but if you really want
>>>> to test CAS updates in the meantime, you'll have to provide the columns in
>>>> (column family comparator) sorted order to the thrift cas() method.
>>>>
>>>> --
>>>> Sylvain
>>>>
>>>>
>>>> On Sun, Jul 21, 2013 at 1:22 PM, Soumava Ghosh <soumava@cs.utexas.edu>wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I'm taking a look at the Check and Set functionalities provided by the
>>>>> cas() API provided by cassandra 2.0 (the code available on git). I'm
>>>>> running a few tests on a small sized cluster (replication factor 3,
>>>>> consistency level quorum) with a few clients. I've observed a lot of
cases
>>>>> seem to hit the TimedOutException while paxos is in progress. On inspection
>>>>> of the server side logs the following stack was seen:
>>>>>
>>>>> ERROR [Thread-582] 2013-07-21 01:17:46,169 CassandraDaemon.java (line
>>>>> 196) Exception in thread Thread[Thread-582,5,main]
>>>>> java.lang.AssertionError: Added column does not sort as the last column
>>>>>         at
>>>>> org.apache.cassandra.db.ArrayBackedSortedColumns.addColumn(ArrayBackedSortedColumns.java:115)
>>>>>         at
>>>>> org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:117)
>>>>>         at
>>>>> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:119)
>>>>>         at
>>>>> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:96)
>>>>>         at
>>>>> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:91)
>>>>>         at
>>>>> org.apache.cassandra.service.paxos.Commit$CommitSerializer.deserialize(Commit.java:139)
>>>>>         at
>>>>> org.apache.cassandra.service.paxos.Commit$CommitSerializer.deserialize(Commit.java:128)
>>>>>         at org.apache.cassandra.net.MessageIn.read(MessageIn.java:99)
>>>>>         at
>>>>> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:175)
>>>>>         at
>>>>> org.apache.cassandra.net.IncomingTcpConnection.handleModernVersion(IncomingTcpConnection.java:135)
>>>>>         at
>>>>> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:82)
>>>>>
>>>>> I was updating multiple columns using the same cas() call, and looking
>>>>> at the assert (code below) I added code to sort them before sending.
>>>>> However, now I'm seeing the same issue *even with single column*cas()
calls.
>>>>>
>>>>>         int c =
>>>>> internalComparator().compare(columns.get(getColumnCount() - 1).name(),
>>>>> column.name());
>>>>>         // note that we want an assertion here (see addColumn
>>>>> javadoc), but we also want that if
>>>>>         // assertion are disabled, addColumn works correctly with
>>>>> unsorted input
>>>>>         assert c <= 0 : "Added column does not sort as the " +
>>>>> (reversed ? "first" : "last") + " column";
>>>>>
>>>>> I have also seen this error occur in the getRow() path, as below:
>>>>>
>>>>> java.lang.AssertionError: Added column does not sort as the last column
>>>>>         at
>>>>> org.apache.cassandra.db.ArrayBackedSortedColumns.addColumn(ArrayBackedSortedColumns.java:115)
>>>>>         at
>>>>> org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:117)
>>>>>         at
>>>>> org.apache.cassandra.db.ColumnFamily.addAtom(ColumnFamily.java:151)
>>>>>         at
>>>>> org.apache.cassandra.db.CollationController.collectTimeOrderedData(CollationController.java:95)
>>>>>         at
>>>>> org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:57)
>>>>>         at
>>>>> org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1452)
>>>>>         at
>>>>> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1281)
>>>>>         at
>>>>> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1195)
>>>>>         at org.apache.cassandra.db.Table.getRow(Table.java:331)
>>>>>         at
>>>>> org.apache.cassandra.db.SliceByNamesReadCommand.getRow(SliceByNamesReadCommand.java:55)
>>>>>         at
>>>>> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1288)
>>>>>         at
>>>>> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1813)
>>>>>         at
>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>>>         at
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>>>         at java.lang.Thread.run(Thread.java:722)
>>>>>
>>>>> Could someone provide me a little more clarity about what the context
>>>>> of the error is and what the client needs to do to fix it? I understand
>>>>> that running with assertions disabled would do away with this error,
but
>>>>> i'd like to know if there is a proper way to fix this. Moreover, what
could
>>>>> possibly be going wrong in the get path that is hitting this issue?
>>>>>
>>>>> Some guidance, on the fact that whether it is expected that the client
>>>>> sort the columns before calling the API with multiple columns, would
be
>>>>> really appreciated.
>>>>>
>>>>> Thanks,
>>>>> Soumava
>>>>>
>>>>
>>>>
>>>
>>
>

Mime
View raw message