Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Message-ID: <4E531B03.3090006@mebigfatguy.com>
Date: Mon, 22 Aug 2011 23:14:11 -0400
From: Dave Brosius <dbrosius@mebigfatguy.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
 rv:1.9.2.17) Gecko/20110424 Thunderbird/3.1.10
MIME-Version: 1.0
To: user@cassandra.apache.org
Subject: Re: Recover from startup problems
References: 
 <CAOA66tHts4nBzsdD3uUSTm22V83wZVJr4xZBRCayMkj6TSWnyA@mail.gmail.com>
 <CALdd-zi5gKgrjpXB_qnF7fsnQiqiOuE25vg_ECChYDzJGQNk2Q@mail.gmail.com>
 <4E51F454.6010107@mebigfatguy.com>
 <CALdd-zi5eW-nACD6LU8svx_aVm50f5NRjkR0A60Zq4vHtdQOLg@mail.gmail.com>
In-Reply-To: 
 <CALdd-zi5eW-nACD6LU8svx_aVm50f5NRjkR0A60Zq4vHtdQOLg@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Sure i'll try that, but I'm pretty sure it was creating a column family 
without any column meta data (types), then, client.insert'ing a 
ByteBuffer that wasn't based on bytes from a String.getBytes call.


On 08/22/2011 11:09 AM, Jonathan Ellis wrote:
> Yes, you can blow away both the data and commitlog directories and
> restart, but can you try these first to troubleshoot?
>
> 1. make a copy of the commitlog directory
> 2. downgrade to 0.8 with no other changes, to see if it's something on
> the new read path
> 2a. if 0.8 starts up then we will fix the read code in trunk
> 2b. if 0.8 doesn't start up either, remove the data directory but NOT
> commitlog, and restart.  this will cause commitlog to be replayed --
> with luck whatever is causing the problem is still in there, so if it
> breaks again, we have a reproducible case
>
> Thanks!
>
> On Mon, Aug 22, 2011 at 1:16 AM, Dave Brosius<dbrosius@mebigfatguy.com>  wrote:
>> Greetings, I'm running head from source, and now when i try to start up the
>> database, i get the following exception which causes client connection
>> failures. I'm fine with blowing away the database, just playing, but wanted
>> to know if there is a safe way to do this.
>>
>> Exception encountered during startup.
>> java.lang.RuntimeException: error reading 1 of 3
>>     at
>> org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:83)
>>     at
>> org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:40)
>>     at
>> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
>>     at
>> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
>>     at
>> org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:107)
>>     at
>> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:194)
>>     at org.apache.cassandra.utils.MergeIterator.<init>(MergeIterator.java:47)
>>     at
>> org.apache.cassandra.utils.MergeIterator$ManyToOne.<init>(MergeIterator.java:142)
>>     at org.apache.cassandra.utils.MergeIterator.get(MergeIterator.java:66)
>>     at
>> org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:96)
>>     at
>> org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:221)
>>     at
>> org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:63)
>>     at
>> org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1285)
>>     at
>> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1169)
>>     at
>> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1120)
>>     at org.apache.cassandra.db.DefsTable.loadFromStorage(DefsTable.java:83)
>>     at
>> org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:507)
>>     at
>> org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:159)
>>     at
>> org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:335)
>>     at
>> org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:91)
>> Caused by: java.nio.channels.ClosedChannelException
>>     at
>> org.apache.cassandra.io.util.RandomAccessReader.read(RandomAccessReader.java:268)
>>     at java.io.RandomAccessFile.readByte(RandomAccessFile.java:589)
>>     at
>> org.apache.cassandra.utils.ByteBufferUtil.readShortLength(ByteBufferUtil.java:356)
>>     at
>> org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:367)
>>     at
>> org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:87)
>>     at
>> org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:82)
>>     at
>> org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:72)
>>     at
>> org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:36)
>>     at
>> org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:79)
>>
>
>