incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Philippe <watche...@gmail.com>
Subject Re: Token != DecoratedKey assertion
Date Wed, 05 Oct 2011 10:22:01 GMT
A little feedback,
I scrubbed on each server and I haven't seen this error again. The load on
each server eems to be correct.
nodetool compactionstats shows  boat-load of "Scrub" at 100% on my 3rd node
but not on the 2 others.
I left it that way and haven't restart yet.

2011/9/26 aaron morton <aaron@thelastpickle.com>

> Looks like a mismatch between the key the index says should be at a certain
> position in the date file and the key that is actually there.
>
> I've not checked but scrub *may* fix this this. Try it and see.
>
> (repair is for repairing consistency between nodes, scrub fixes local
> issues with data. )
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 26/09/2011, at 12:53 PM, Philippe wrote:
>
> Juste did
> Could there be data corruption or will repairs do this?
>
> Thanks
> Le 25 sept. 2011 15:30, "Jonathan Ellis" <jbellis@gmail.com> a écrit :
> > Assertion errors are bugs, so that should worry you.
> >
> > However, I'd upgrade before filing a ticket. There were a lot of
> > fixes in 0.8.5.
> >
> > On Sun, Sep 25, 2011 at 2:27 AM, Philippe <watcherfr@gmail.com> wrote:
> >> Hello,
> >> I've seen a couple of these in my logs, running 0.8.4.
> >> This is a RF=3, 3-node cluster. 2 nodes including this one are on 0.8.4
> and
> >> one is on 0.8.5
> >>
> >> The node is still functionning hours later. Should I be worried ?
> >>
> >> Thanks
> >>
> >> ERROR [ReadStage:94911] 2011-09-24 22:40:30,043
> AbstractCassandraDaemon.java
> >> (line 134) Fatal exception in thread Thread[ReadStage:94911,5,main]
> >> java.lang.AssertionError:
> >>
> DecoratedKey(Token(bytes[0000224ceb80b5fb11e00000848783ceb9bf0000000000000002ffffffffffffff33]),
> >> 0000224ceb80b5fb11e00000848783ceb9bf0000000000000002ffffffffffffff33) !=
> >> DecoratedKey(Token(bytes[038453154cb00000005f14]),
> 038453154cb00000005f14)
> >> in /var/lib/cassandra/data/XXXXXXXXX/PUBLIC_MONTHLY_20-g-10634-Data.db
> >>         at
> >>
> org.apache.cassandra.db.columniterator.SSTableSliceIterator.<init>(SSTableSliceIterator.java:59)
> >>         at
> >>
> org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:66)
> >>         at
> >>
> org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:80)
> >>         at
> >>
> org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1315)
> >>         at
> >>
> org.apache.cassandra.db.ColumnFamilyStore.cacheRow(ColumnFamilyStore.java:1182)
> >>         at
> >>
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1222)
> >>         at
> >>
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1169)
> >>         at org.apache.cassandra.db.Table.getRow(Table.java:385)
> >>         at
> >>
> org.apache.cassandra.db.SliceByNamesReadCommand.getRow(SliceByNamesReadCommand.java:58)
> >>         at
> >>
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:642)
> >>         at
> >>
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1107)
> >>         at
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >>         at
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >>         at java.lang.Thread.run(Thread.java:662)
> >> ERROR [ReadStage:94936] 2011-09-24 22:40:30,042
> AbstractCassandraDaemon.java
> >> (line 134) Fatal exception in thread Thread[ReadStage:94936,5,main]
> >> java.lang.AssertionError: DecoratedKey(Token(bytes[]), ) !=
> >> DecoratedKey(Token(bytes[038453154c900000005f14]),
> 038453154c900000005f14)
> >> in /var/lib/cassandra/data/XXXXXXXXX/PUBLIC_MONTHLY_20-g-10634-Data.db
> >>         at
> >>
> org.apache.cassandra.db.columniterator.SSTableSliceIterator.<init>(SSTableSliceIterator.java:59)
> >>         at
> >>
> org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:66)
> >>         at
> >>
> org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:80)
> >>         at
> >>
> org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1315)
> >>         at
> >>
> org.apache.cassandra.db.ColumnFamilyStore.cacheRow(ColumnFamilyStore.java:1182)
> >>         at
> >>
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1222)
> >>         at
> >>
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1169)
> >>         at org.apache.cassandra.db.Table.getRow(Table.java:385)
> >>         at
> >>
> org.apache.cassandra.db.SliceByNamesReadCommand.getRow(SliceByNamesReadCommand.java:58)
> >>         at
> >>
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:642)
> >>         at
> >>
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1107)
> >>         at
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >>         at
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >>         at java.lang.Thread.run(Thread.java:662)
> >> ERROR [ReadStage:94713] 2011-09-24 22:40:30,041
> AbstractCassandraDaemon.java
> >> (line 134) Fatal exception in thread Thread[ReadStage:94713,5,main]
> >> java.lang.AssertionError:
> >>
> DecoratedKey(Token(bytes[7c4831fe0000000000000001ffffffffffffffaa000c6c697665626f782d6265306580000000800000000000000000000002000700031195010000048000000000000000000001327e62362a00000024000100009c0dc550c60111e0000001687c4831fe00000000000000010000000000000001000700031195010001048000000000000000000001327e62362a00000024000100009c0dc550c60111e0000001687c4831fe0000000000000001ffffffffffffffb0000c6c697665626f782d6631326380000000800000000000000000000002000700031195010000048000000000000000000001327e62362a00000024000100009c0dc550c60111e0000001687c4831fe00000000000000010000000000000001000700031195010001048000000000000000000001327e62362a00000024000100009c0dc550c60111e0000001687c4831fe0000000000000001ffffffffffffffab000b062a27f9e3000000005f13000000]),
> >>
> 7c4831fe0000000000000001ffffffffffffffaa000c6c697665626f782d6265306580000000800000000000000000000002000700031195010000048000000000000000000001327e62362a00000024000100009c0dc550c60111e0000001687c4831fe00000000000000010000000000000001000700031195010001048000000000000000000001327e62362a00000024000100009c0dc550c60111e0000001687c4831fe0000000000000001ffffffffffffffb0000c6c697665626f782d6631326380000000800000000000000000000002000700031195010000048000000000000000000001327e62362a00000024000100009c0dc550c60111e0000001687c4831fe00000000000000010000000000000001000700031195010001048000000000000000000001327e62362a00000024000100009c0dc550c60111e0000001687c4831fe0000000000000001ffffffffffffffab000b062a27f9e3000000005f13000000)
> >> != DecoratedKey(Token(bytes[038451bfeeb00000005f14]),
> >> 038451bfeeb00000005f14) in
> >> /var/lib/cassandra/data/XXXXXXXXX/PUBLIC_MONTHLY_20-g-12305-Data.db
> >>
> >>
> >>
> >>
> >
> >
> >
> > --
> > Jonathan Ellis
> > Project Chair, Apache Cassandra
> > co-founder of DataStax, the source for professional Cassandra support
> > http://www.datastax.com
>
>
>

Mime
View raw message