incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stu King <...@stuartrexking.com>
Subject Move
Date Fri, 04 Feb 2011 08:18:10 GMT
I am running a move on one node in a 5 node cluster. There are no writes to
the cluster during the move.

I am seeing an exception on one of the nodes (not the node which I am doing
the move on).

The exception stack is

ERROR [CompactionExecutor:1] 2011-02-04 08:10:46,855 PrecompactedRow.java
(line 82) Skipping row DecoratedKey(656517988577125179070965247963445,
555345524e414d452e6a6f746173696c766573747265) in
/var/lib/cassandra/data/Wenzani/UUID_UUID_SUPER-e-408-Data.db
java.io.EOFException
at java.io.RandomAccessFile.readFully(RandomAccessFile.java:416)
at
org.apache.cassandra.utils.FBUtilities.readByteArray(FBUtilities.java:280)
at
org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:94)
at
org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:364)
at
org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:313)
at
org.apache.cassandra.db.ColumnFamilySerializer.deserializeColumns(ColumnFamilySerializer.java:129)
at
org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:137)
at org.apache.cassandra.io.PrecompactedRow.<init>(PrecompactedRow.java:78)
at
org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:138)
at
org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:107)
at
org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:42)
at
org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
at
org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
at
org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
at
org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:323)
at
org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:122)
at
org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:92)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)

Output from nodetool ring.

Address         Status State   Load            Owns    Token


105916716988735575505223832861775432335
1.1.1.2   Up     Normal  34.29 GB        45.36%
 12956529933298582072612274413196299151
1.1.1.3    Up     Normal  34.46 GB        11.41%
 32366675628954067180152712803029297247
1.1.1.4   Up     Normal  48.96 GB        11.40%
 51756081624280481651195537730585467204
1.1.1.5  Up     Normal  22 GB           22.78%
 90515859237527157456212262236145255573
1.1.1.6   Up     Leaving 13.34 GB        9.05%
105916716988735575505223832861775432335

1.1.1.6 is the node which I executed the move on. It seems to be locked in
the Leaving state. Is this normal until the move completes?

There is almost no activity in the logs and very little cpu usage across the
cluster.

Is this expected for a move?

Cheers

Stu

Mime
View raw message