incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sylvain Lebresne <sylv...@datastax.com>
Subject Re: Reboot, now node down 0.8rc1
Date Tue, 24 May 2011 12:53:00 GMT
Do you have a one of your super column family with a fairly short
value for gc_grace_seconds ?

I (strongly) suspect you're hitting
https://issues.apache.org/jira//browse/CASSANDRA-2675.

--
Sylvain

On Tue, May 24, 2011 at 2:29 PM, Scott McPheeters
<smcpheeters@healthx.com> wrote:
> It was a 0.8beta2 install that last week I upgraded to rc1.
>
> I will turn the logging up to debug the next time I have issues to get
> more details.
>
> Thank you.
>
>
>
> Scott
>
> -----Original Message-----
> From: aaron morton [mailto:aaron@thelastpickle.com]
> Sent: Monday, May 23, 2011 6:42 PM
> To: user@cassandra.apache.org
> Subject: Re: Reboot, now node down 0.8rc1
>
> You could have removed the affected commit log file and then run a
> nodetool repair after the node had started.
>
> It would be handy to have some more context for the problem. Was this an
> upgrade from 0.7 or a fresh install?
>
> If you are running the rc's it's handy to turn logging up to DEBUG so
> there are some more details if things go wrong.
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 24 May 2011, at 08:05, Scott McPheeters wrote:
>
>> Since this is a testing system, I deleted the commit log and it came
>> right up.  My question now is, let's say I had a ton of data in the
>> commit log that this node needs now.  What is the best way to get the
>> data back to the node?  Does a nodetool repair do this? Or do I need
> to
>> decommission the node and bring it back?  Or am I missing completely
>> what the commitlog is?
>>
>>
>> Scott
>>
>>
>> -----Original Message-----
>> From: Scott McPheeters [mailto:smcpheeters@healthx.com]
>> Sent: Monday, May 23, 2011 2:18 PM
>> To: user@cassandra.apache.org
>> Subject: Reboot, now node down 0.8rc1
>>
>> I have a test node system running release 0.8rc1.  I rebooted node3
> and
>> now Cassandra is failing on startup.
>>
>> Any ideas?  I am not sure where to begin.
>>
>> Debian 6, plenty of disk space, Cassandra 0.8rc1
>>
>>
>> INFO 13:48:58,192 Creating new commitlog segment
>> /home/cassandra/commitlog/CommitLog-1306172938192.log
>> INFO 13:48:58,236 Replaying
>> /home/cassandra/commitlog/CommitLog-1305918923361.log
>> INFO 13:49:04,041 Finished reading
>> /home/cassandra/commitlog/CommitLog-1305918923361.log
>> java.lang.reflect.InvocationTargetException
>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>        at
>>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav
>> a:39)
>>        at
>>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
>> Impl.java:25)
>>        at java.lang.reflect.Method.invoke(Method.java:597)
>>        at
>>
> org.apache.commons.daemon.support.DaemonLoader.load(DaemonLoader.java:16
>> 0)
>> Caused by: java.io.IOError: java.io.EOFException
>>        at
>>
> org.apache.cassandra.io.util.ColumnIterator.deserializeNext(ColumnSorted
>> Map.java:265)
>>        at
>>
> org.apache.cassandra.io.util.ColumnIterator.next(ColumnSortedMap.java:28
>> 1)
>>        at
>>
> org.apache.cassandra.io.util.ColumnIterator.next(ColumnSortedMap.java:23
>> 6)
>>        at
>>
> java.util.concurrent.ConcurrentSkipListMap.buildFromSorted(ConcurrentSki
>> pListMap.java:1493)
>>        at
>>
> java.util.concurrent.ConcurrentSkipListMap.<init>(ConcurrentSkipListMap.
>> java:1443)
>>        at
>>
> org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.ja
>> va:402)
>>        at
>>
> org.apache.cassandra.db.ColumnFamilySerializer.deserializeColumns(Column
>> FamilySerializer.java:136)
>>        at
>>
> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilyS
>> erializer.java:126)
>>        at
>>
> org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(Ro
>> wMutation.java:367)
>>        at
>>
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:271)
>>        at
>>
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:158)
>>        at
>>
> org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassa
>> ndraDaemon.java:174)
>>        at
>>
> org.apache.cassandra.service.AbstractCassandraDaemon.init(AbstractCassan
>> draDaemon.java:216)
>>        ... 5 more
>> Caused by: java.io.EOFException
>>        at java.io.DataInputStream.readFully(DataInputStream.java:180)
>>        at java.io.DataInputStream.readFully(DataInputStream.java:152)
>>        at
>>
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:394)
>>        at
>>
> org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBuffer
>> Util.java:368)
>>        at
>>
> org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.ja
>> va:87)
>>        at
>>
> org.apache.cassandra.io.util.ColumnIterator.deserializeNext(ColumnSorted
>> Map.java:261)
>>        ... 17 more
>> Cannot load daemon
>> Service exit with a return value of 3
>>
>>
>> Scott
>>
>
>

Mime
View raw message