incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Colin Kuo <colinkuo...@gmail.com>
Subject Starting up Cassandra occurred errors after upgrading Cassandra to 1.2.5 from 1.0.12
Date Wed, 29 May 2013 07:32:26 GMT
Hi All,

We followed the upgrade guide(
http://www.datastax.com/docs/1.2/install/upgrading) from Datastax web site
and upgraded Cassadra to 1.2.5, but it occurred errors in system.log when
starting up.

After digging into code level, it looks like Cassandra found the file
length of IndexSummary sstable is zero. Thus Cassandra threw
AssertionError. In fact, the file length of the IndexSummary is about 80
bytes, not zero. It's weird.

Also we observed that only happens on the IndexSummary file of secondary
index. The errors can be reproducible. Below are my upgrade steps.
1. Shutdown all of client applications.
2. Run "nodetool drain" before shutting down the existing Cassandra service.
3. Stop old Cassandra process, then start the new binary process using
migrated cassandra.yaml.
4. Run "nodetool upgradesstables -a" in order to upgrade all of sstable
files become new format.
5. Restart Cassandra process and monitor the logs file for any issues.
At step 5, we found the error messages as below.

Any ideas?

Thank you!
Colin

===================================================================================================================================
 INFO [SSTableBatchOpen:2] 2013-05-29 04:38:40,085 SSTableReader.java (line
169) Opening
/var/lib/cassandra/data/ks/user/ks-user.ks_user_personalID-ic-61 (58 bytes)
ERROR [SSTableBatchOpen:1] 2013-05-29 04:38:40,085 CassandraDaemon.java
(line 175) Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.AssertionError
at
org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:401)
at
org.apache.cassandra.io.sstable.IndexSummary$IndexSummarySerializer.deserialize(IndexSummary.java:124)
 at
org.apache.cassandra.io.sstable.SSTableReader.loadSummary(SSTableReader.java:426)
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:360)
 at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:201)
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:154)
 at
org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.java:241)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
 at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
 at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
 at java.lang.Thread.run(Unknown Source)
ERROR [SSTableBatchOpen:2] 2013-05-29 04:38:40,085 CassandraDaemon.java
(line 175) Exception in thread Thread[SSTableBatchOpen:2,5,main]
java.lang.AssertionError
at
org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:401)
at
org.apache.cassandra.io.sstable.IndexSummary$IndexSummarySerializer.deserialize(IndexSummary.java:124)
 at
org.apache.cassandra.io.sstable.SSTableReader.loadSummary(SSTableReader.java:426)
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:360)
 at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:201)
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:154)
 at
org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.java:241)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
 at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
 at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
 at java.lang.Thread.run(Unknown Source)
===================================================================================================================================

Mime
View raw message