incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Freeman <8fo...@gmail.com>
Subject Re: FileNotFoundException while inserting (1.2.8)
Date Wed, 11 Sep 2013 13:49:11 GMT
Yes, I started with a fresh keyspace (dropped and re-created) to run 
this test.

On 09/10/2013 02:01 PM, sankalp kohli wrote:
> Have you dropped and recreated a keyspace with the same name recently?
>
>
> On Tue, Sep 10, 2013 at 8:40 AM, Keith Freeman <8forty@gmail.com 
> <mailto:8forty@gmail.com>> wrote:
>
>     While running a heavy insert load, one of my nodes started
>     throwing this exception when trying a compaction:
>
>      INFO [CompactionExecutor:23] 2013-09-09 16:08:07,528
>     CompactionTask.java (line 105) Compacting [SSTableReader(p
>     ath='/var/lib/cassandra/data/smdb/tracedata/smdb-tracedata-ic-6-Data.db'),
>     SSTableReader(path='/var/lib/cassandr
>     a/data/smdb/tracedata/smdb-tracedata-ic-5-Data.db'),
>     SSTableReader(path='/var/lib/cassandra/data/smdb/tracedata/
>     smdb-tracedata-ic-1-Data.db'),
>     SSTableReader(path='/var/lib/cassandra/data/smdb/tracedata/smdb-tracedata-ic-4-Da
>     ta.db'),
>     SSTableReader(path='/var/lib/cassandra/data/smdb/tracedata/smdb-tracedata-ic-2-Data.db')]
>     ERROR [CompactionExecutor:23] 2013-09-09 16:08:07,611
>     CassandraDaemon.java (line 192) Exception in thread Thread
>     [CompactionExecutor:23,1,main]
>     java.lang.RuntimeException: java.io.FileNotFoundException:
>     /var/lib/cassandra/data/smdb/tracedata/smdb-tracedata
>     -ic-5-Data.db (No such file or directory)
>             at
>     org.apache.cassandra.io.util.ThrottledReader.open(ThrottledReader.java:53)
>             at org.apache.cassandra.io
>     <http://org.apache.cassandra.io>.sstable.SSTableReader.openDataReader(SSTableReader.java:1194)
>             at org.apache.cassandra.io
>     <http://org.apache.cassandra.io>.sstable.SSTableScanner.<init>(SSTableScanner.java:54)
>             at org.apache.cassandra.io
>     <http://org.apache.cassandra.io>.sstable.SSTableReader.getDirectScanner(SSTableReader.java:1014)
>             at org.apache.cassandra.io
>     <http://org.apache.cassandra.io>.sstable.SSTableReader.getDirectScanner(SSTableReader.java:1026)
>             at
>     org.apache.cassandra.db.compaction.AbstractCompactionStrategy.getScanners(AbstractCompactionStrategy.
>     java:157)
>             at
>     org.apache.cassandra.db.compaction.AbstractCompactionStrategy.getScanners(AbstractCompactionStrategy.java:163)
>             at
>     org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:117)
>             at
>     org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
>             at
>     org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>             at
>     org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:58)
>             at
>     org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60)
>             at
>     org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:211)
>             at
>     java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>             at
>     java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>             at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>             at
>     java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>             at
>     java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>             at java.lang.Thread.run(Thread.java:662)
>     Caused by: java.io.FileNotFoundException:
>     /var/lib/cassandra/data/smdb/tracedata/smdb-tracedata-ic-5-Data.db
>     (No such file or directory)
>             at java.io.RandomAccessFile.open(Native Method)
>             at java.io.RandomAccessFile.<init>(RandomAccessFile.java:216)
>             at
>     org.apache.cassandra.io.util.RandomAccessReader.<init>(RandomAccessReader.java:67)
>             at
>     org.apache.cassandra.io.util.ThrottledReader.<init>(ThrottledReader.java:35)
>             at
>     org.apache.cassandra.io.util.ThrottledReader.open(ThrottledReader.java:49)
>             ... 18 more
>
>     This shows up many times in the log.  I figured running a repair
>     on the node might fix it, but the repair ran for over an hour (the
>     node only has about 3G of data), so I figured it was hung.  I
>     tried restarting the repair, but each time it starts the node logs
>     that same exception immediately:
>
>      INFO [AntiEntropySessions:5] 2013-09-10 09:36:35,526
>     AntiEntropyService.java (line 651) [repair #c6ab9c00-1a2e-
>     11e3-b0e5-05d1729cecff] new session: will sync /192.168.27.73
>     <http://192.168.27.73>, /192.168.27.75 <http://192.168.27.75> on
>     range (4925454539472655923,4991
>     066214171147775] for smdb.[tracedata, processors]
>      INFO [AntiEntropySessions:5] 2013-09-10 09:36:35,526
>     AntiEntropyService.java (line 857) [repair #c6ab9c00-1a2e-
>     11e3-b0e5-05d1729cecff] requesting merkle trees for tracedata (to
>     [/192.168.27.75 <http://192.168.27.75>, /192.168.27.73
>     <http://192.168.27.73>])
>     ERROR [ValidationExecutor:2] 2013-09-10 09:36:35,535
>     CassandraDaemon.java (line 192) Exception in thread Thread[
>     ValidationExecutor:2,1,main]
>     java.lang.RuntimeException: java.io.FileNotFoundException:
>     /var/lib/cassandra/data/smdb/tracedata/smdb-tracedata
>     -ic-5-Data.db (No such file or directory)
>             at
>     org.apache.cassandra.io.util.ThrottledReader.open(ThrottledReader.java:53)
>             at org.apache.cassandra.io
>     <http://org.apache.cassandra.io>.sstable.SSTableReader.openDataReader(SSTableReader.java:1194)
>             ...
>
>     How can I fix this node?
>
>
>
>


Mime
View raw message