cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joaquin Casares (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CASSANDRA-7308) Dropped keyspace causes issues later
Date Tue, 27 May 2014 22:54:02 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-7308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Joaquin Casares updated CASSANDRA-7308:
---------------------------------------

    Description: 
I had a keyspace that was accepting writes and was able to query the keyspace via cqlsh. I
re-ran my test application that drops the keyspace if it exists then creates the keyspace
and wrote some more. This time, however, any select request via cqlsh breaks with a `Request
did not complete within rpc_timeout.`.

This was seen in the system.log:

{CODE}
ERROR [ReadStage:61096] 2014-05-27 17:21:39,995 CassandraDaemon.java (line 198) Exception
in thread Thread[ReadStage:61096,5,main]
java.lang.RuntimeException: java.lang.RuntimeException: java.io.FileNotFoundException: /var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-36-Data.db
(No such file or directory)
	at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1927)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: /var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-36-Data.db
(No such file or directory)
	at org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:59)
	at org.apache.cassandra.io.sstable.SSTableReader.openDataReader(SSTableReader.java:1362)
	at org.apache.cassandra.io.sstable.SSTableScanner.<init>(SSTableScanner.java:67)
	at org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1147)
	at org.apache.cassandra.db.RowIteratorFactory.getIterator(RowIteratorFactory.java:69)
	at org.apache.cassandra.db.ColumnFamilyStore.getSequentialIterator(ColumnFamilyStore.java:1579)
	at org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1699)
	at org.apache.cassandra.db.RangeSliceCommand.executeLocally(RangeSliceCommand.java:137)
	at org.apache.cassandra.service.StorageProxy$LocalRangeSliceRunnable.runMayThrow(StorageProxy.java:1381)
	at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1923)
	... 3 more
Caused by: java.io.FileNotFoundException: /var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-36-Data.db
(No such file or directory)
	at java.io.RandomAccessFile.open(Native Method)
	at java.io.RandomAccessFile.<init>(RandomAccessFile.java:241)
	at org.apache.cassandra.io.util.RandomAccessReader.<init>(RandomAccessReader.java:58)
	at org.apache.cassandra.io.compress.CompressedRandomAccessReader.<init>(CompressedRandomAccessReader.java:76)
	at org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:55)
	... 12 more
 INFO [CompactionExecutor:708] 2014-05-27 17:21:42,810 ColumnFamilyStore.java (line 787) Enqueuing
flush of Memtable-compactions_in_progress@46337516(0/0 serialized/live bytes, 1 ops)
 WARN [MemoryMeter:1] 2014-05-27 17:21:42,810 Memtable.java (line 470) setting live ratio
to maximum of 64.0 instead of Infinity
 INFO [MemoryMeter:1] 2014-05-27 17:21:42,811 Memtable.java (line 481) CFS(Keyspace='system',
ColumnFamily='compactions_in_progress') liveRatio is 64.0 (just-counted was 64.0).  calculation
took 0ms for 0 cells
 INFO [FlushWriter:92] 2014-05-27 17:21:42,811 Memtable.java (line 363) Writing Memtable-compactions_in_progress@46337516(0/0
serialized/live bytes, 1 ops)
 INFO [FlushWriter:92] 2014-05-27 17:21:42,817 Memtable.java (line 403) Completed flushing
/var/lib/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-jb-51-Data.db
(42 bytes) for commitlog position ReplayPosition(segmentId=1400882649502, position=5391795)
 INFO [CompactionExecutor:708] 2014-05-27 17:21:42,819 CompactionTask.java (line 275) Compacted
4 sstables to [/var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-87,].
 56,962,552 bytes to 56,961,531 (~99% of original) in 77,261ms = 0.703107MB/s.  5 total partitions
merged to 2.  Partition merge counts were {1:1, 4:1, }
 INFO [CompactionExecutor:708] 2014-05-27 17:21:42,820 ColumnFamilyStore.java (line 787) Enqueuing
flush of Memtable-compactions_in_progress@1777281623(179/3463 serialized/live bytes, 9 ops)
 INFO [MemoryMeter:1] 2014-05-27 17:21:42,820 Memtable.java (line 481) CFS(Keyspace='system',
ColumnFamily='compactions_in_progress') liveRatio is 17.862604748603353 (just-counted was
16.374301675977655).  calculation took 0ms for 7 cells
 INFO [FlushWriter:92] 2014-05-27 17:21:42,820 Memtable.java (line 363) Writing Memtable-compactions_in_progress@1777281623(179/3197
serialized/live bytes, 9 ops)
 INFO [FlushWriter:92] 2014-05-27 17:21:42,829 Memtable.java (line 403) Completed flushing
/var/lib/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-jb-52-Data.db
(168 bytes) for commitlog position ReplayPosition(segmentId=1400882649502, position=5392718)
 INFO [CompactionExecutor:706] 2014-05-27 17:21:42,829 CompactionTask.java (line 115) Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-jb-51-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-jb-50-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-jb-52-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-jb-49-Data.db')]
 INFO [CompactionExecutor:708] 2014-05-27 17:21:42,830 CompactionTask.java (line 115) Compacting
[SSTableReader(path='/var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-71-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-81-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-76-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-87-Data.db')]
 INFO [CompactionExecutor:708] 2014-05-27 17:21:42,892 CompactionController.java (line 192)
Compacting large row weathercql/historical:SFJ (527238028 bytes) incrementally
 INFO [CompactionExecutor:706] 2014-05-27 17:21:42,894 CompactionTask.java (line 275) Compacted
4 sstables to [/var/lib/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-jb-53,].
 607 bytes to 321 (~52% of original) in 64ms = 0.004783MB/s.  8 total partitions merged to
4.  Partition merge counts were {1:4, 2:2, }
 INFO [CompactionExecutor:710] 2014-05-27 17:21:57,469 CompactionController.java (line 192)
Compacting large row weathercql/historical:UAK (205611614 bytes) incrementally
ERROR [ReadStage:61106] 2014-05-27 17:22:20,563 CassandraDaemon.java (line 198) Exception
in thread Thread[ReadStage:61106,5,main]
java.lang.RuntimeException: java.lang.RuntimeException: java.io.FileNotFoundException: /var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-36-Data.db
(No such file or directory)
	at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1927)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: /var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-36-Data.db
(No such file or directory)
	at org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:59)
	at org.apache.cassandra.io.sstable.SSTableReader.openDataReader(SSTableReader.java:1362)
	at org.apache.cassandra.io.sstable.SSTableScanner.<init>(SSTableScanner.java:67)
	at org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1147)
	at org.apache.cassandra.db.RowIteratorFactory.getIterator(RowIteratorFactory.java:69)
	at org.apache.cassandra.db.ColumnFamilyStore.getSequentialIterator(ColumnFamilyStore.java:1579)
	at org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1699)
	at org.apache.cassandra.db.RangeSliceCommand.executeLocally(RangeSliceCommand.java:137)
	at org.apache.cassandra.service.StorageProxy$LocalRangeSliceRunnable.runMayThrow(StorageProxy.java:1381)
	at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1923)
	... 3 more
Caused by: java.io.FileNotFoundException: /var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-36-Data.db
(No such file or directory)
	at java.io.RandomAccessFile.open(Native Method)
	at java.io.RandomAccessFile.<init>(RandomAccessFile.java:241)
	at org.apache.cassandra.io.util.RandomAccessReader.<init>(RandomAccessReader.java:58)
	at org.apache.cassandra.io.compress.CompressedRandomAccessReader.<init>(CompressedRandomAccessReader.java:76)
	at org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:55)
	... 12 more
 INFO [CompactionExecutor:707] 2014-05-27 17:22:20,661 CompactionController.java (line 192)
Compacting large row weathercql/historical:MAG (477808828 bytes) incrementally
 INFO [CompactionExecutor:710] 2014-05-27 17:22:34,154 CompactionController.java (line 192)
Compacting large row weathercql/historical:MAG (537030028 bytes) incrementally
{CODE}

  was:
I had a keyspace that was accepting writes and was able to query the keyspace via cqlsh. I
re-ran my test application that drops the keyspace if it exists and wrote some more. This
time, however, any select request via cqlsh breaks with a `Request did not complete within
rpc_timeout.`.

This was seen in the system.log:

{CODE}
ERROR [ReadStage:61096] 2014-05-27 17:21:39,995 CassandraDaemon.java (line 198) Exception
in thread Thread[ReadStage:61096,5,main]
java.lang.RuntimeException: java.lang.RuntimeException: java.io.FileNotFoundException: /var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-36-Data.db
(No such file or directory)
	at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1927)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: /var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-36-Data.db
(No such file or directory)
	at org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:59)
	at org.apache.cassandra.io.sstable.SSTableReader.openDataReader(SSTableReader.java:1362)
	at org.apache.cassandra.io.sstable.SSTableScanner.<init>(SSTableScanner.java:67)
	at org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1147)
	at org.apache.cassandra.db.RowIteratorFactory.getIterator(RowIteratorFactory.java:69)
	at org.apache.cassandra.db.ColumnFamilyStore.getSequentialIterator(ColumnFamilyStore.java:1579)
	at org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1699)
	at org.apache.cassandra.db.RangeSliceCommand.executeLocally(RangeSliceCommand.java:137)
	at org.apache.cassandra.service.StorageProxy$LocalRangeSliceRunnable.runMayThrow(StorageProxy.java:1381)
	at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1923)
	... 3 more
Caused by: java.io.FileNotFoundException: /var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-36-Data.db
(No such file or directory)
	at java.io.RandomAccessFile.open(Native Method)
	at java.io.RandomAccessFile.<init>(RandomAccessFile.java:241)
	at org.apache.cassandra.io.util.RandomAccessReader.<init>(RandomAccessReader.java:58)
	at org.apache.cassandra.io.compress.CompressedRandomAccessReader.<init>(CompressedRandomAccessReader.java:76)
	at org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:55)
	... 12 more
 INFO [CompactionExecutor:708] 2014-05-27 17:21:42,810 ColumnFamilyStore.java (line 787) Enqueuing
flush of Memtable-compactions_in_progress@46337516(0/0 serialized/live bytes, 1 ops)
 WARN [MemoryMeter:1] 2014-05-27 17:21:42,810 Memtable.java (line 470) setting live ratio
to maximum of 64.0 instead of Infinity
 INFO [MemoryMeter:1] 2014-05-27 17:21:42,811 Memtable.java (line 481) CFS(Keyspace='system',
ColumnFamily='compactions_in_progress') liveRatio is 64.0 (just-counted was 64.0).  calculation
took 0ms for 0 cells
 INFO [FlushWriter:92] 2014-05-27 17:21:42,811 Memtable.java (line 363) Writing Memtable-compactions_in_progress@46337516(0/0
serialized/live bytes, 1 ops)
 INFO [FlushWriter:92] 2014-05-27 17:21:42,817 Memtable.java (line 403) Completed flushing
/var/lib/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-jb-51-Data.db
(42 bytes) for commitlog position ReplayPosition(segmentId=1400882649502, position=5391795)
 INFO [CompactionExecutor:708] 2014-05-27 17:21:42,819 CompactionTask.java (line 275) Compacted
4 sstables to [/var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-87,].
 56,962,552 bytes to 56,961,531 (~99% of original) in 77,261ms = 0.703107MB/s.  5 total partitions
merged to 2.  Partition merge counts were {1:1, 4:1, }
 INFO [CompactionExecutor:708] 2014-05-27 17:21:42,820 ColumnFamilyStore.java (line 787) Enqueuing
flush of Memtable-compactions_in_progress@1777281623(179/3463 serialized/live bytes, 9 ops)
 INFO [MemoryMeter:1] 2014-05-27 17:21:42,820 Memtable.java (line 481) CFS(Keyspace='system',
ColumnFamily='compactions_in_progress') liveRatio is 17.862604748603353 (just-counted was
16.374301675977655).  calculation took 0ms for 7 cells
 INFO [FlushWriter:92] 2014-05-27 17:21:42,820 Memtable.java (line 363) Writing Memtable-compactions_in_progress@1777281623(179/3197
serialized/live bytes, 9 ops)
 INFO [FlushWriter:92] 2014-05-27 17:21:42,829 Memtable.java (line 403) Completed flushing
/var/lib/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-jb-52-Data.db
(168 bytes) for commitlog position ReplayPosition(segmentId=1400882649502, position=5392718)
 INFO [CompactionExecutor:706] 2014-05-27 17:21:42,829 CompactionTask.java (line 115) Compacting
[SSTableReader(path='/var/lib/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-jb-51-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-jb-50-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-jb-52-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-jb-49-Data.db')]
 INFO [CompactionExecutor:708] 2014-05-27 17:21:42,830 CompactionTask.java (line 115) Compacting
[SSTableReader(path='/var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-71-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-81-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-76-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-87-Data.db')]
 INFO [CompactionExecutor:708] 2014-05-27 17:21:42,892 CompactionController.java (line 192)
Compacting large row weathercql/historical:SFJ (527238028 bytes) incrementally
 INFO [CompactionExecutor:706] 2014-05-27 17:21:42,894 CompactionTask.java (line 275) Compacted
4 sstables to [/var/lib/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-jb-53,].
 607 bytes to 321 (~52% of original) in 64ms = 0.004783MB/s.  8 total partitions merged to
4.  Partition merge counts were {1:4, 2:2, }
 INFO [CompactionExecutor:710] 2014-05-27 17:21:57,469 CompactionController.java (line 192)
Compacting large row weathercql/historical:UAK (205611614 bytes) incrementally
ERROR [ReadStage:61106] 2014-05-27 17:22:20,563 CassandraDaemon.java (line 198) Exception
in thread Thread[ReadStage:61106,5,main]
java.lang.RuntimeException: java.lang.RuntimeException: java.io.FileNotFoundException: /var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-36-Data.db
(No such file or directory)
	at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1927)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: /var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-36-Data.db
(No such file or directory)
	at org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:59)
	at org.apache.cassandra.io.sstable.SSTableReader.openDataReader(SSTableReader.java:1362)
	at org.apache.cassandra.io.sstable.SSTableScanner.<init>(SSTableScanner.java:67)
	at org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1147)
	at org.apache.cassandra.db.RowIteratorFactory.getIterator(RowIteratorFactory.java:69)
	at org.apache.cassandra.db.ColumnFamilyStore.getSequentialIterator(ColumnFamilyStore.java:1579)
	at org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1699)
	at org.apache.cassandra.db.RangeSliceCommand.executeLocally(RangeSliceCommand.java:137)
	at org.apache.cassandra.service.StorageProxy$LocalRangeSliceRunnable.runMayThrow(StorageProxy.java:1381)
	at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1923)
	... 3 more
Caused by: java.io.FileNotFoundException: /var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-36-Data.db
(No such file or directory)
	at java.io.RandomAccessFile.open(Native Method)
	at java.io.RandomAccessFile.<init>(RandomAccessFile.java:241)
	at org.apache.cassandra.io.util.RandomAccessReader.<init>(RandomAccessReader.java:58)
	at org.apache.cassandra.io.compress.CompressedRandomAccessReader.<init>(CompressedRandomAccessReader.java:76)
	at org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:55)
	... 12 more
 INFO [CompactionExecutor:707] 2014-05-27 17:22:20,661 CompactionController.java (line 192)
Compacting large row weathercql/historical:MAG (477808828 bytes) incrementally
 INFO [CompactionExecutor:710] 2014-05-27 17:22:34,154 CompactionController.java (line 192)
Compacting large row weathercql/historical:MAG (537030028 bytes) incrementally
{CODE}


> Dropped keyspace causes issues later
> ------------------------------------
>
>                 Key: CASSANDRA-7308
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7308
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Joaquin Casares
>              Labels: datastax_qa
>
> I had a keyspace that was accepting writes and was able to query the keyspace via cqlsh.
I re-ran my test application that drops the keyspace if it exists then creates the keyspace
and wrote some more. This time, however, any select request via cqlsh breaks with a `Request
did not complete within rpc_timeout.`.
> This was seen in the system.log:
> {CODE}
> ERROR [ReadStage:61096] 2014-05-27 17:21:39,995 CassandraDaemon.java (line 198) Exception
in thread Thread[ReadStage:61096,5,main]
> java.lang.RuntimeException: java.lang.RuntimeException: java.io.FileNotFoundException:
/var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-36-Data.db (No such
file or directory)
> 	at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1927)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:724)
> Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: /var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-36-Data.db
(No such file or directory)
> 	at org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:59)
> 	at org.apache.cassandra.io.sstable.SSTableReader.openDataReader(SSTableReader.java:1362)
> 	at org.apache.cassandra.io.sstable.SSTableScanner.<init>(SSTableScanner.java:67)
> 	at org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1147)
> 	at org.apache.cassandra.db.RowIteratorFactory.getIterator(RowIteratorFactory.java:69)
> 	at org.apache.cassandra.db.ColumnFamilyStore.getSequentialIterator(ColumnFamilyStore.java:1579)
> 	at org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1699)
> 	at org.apache.cassandra.db.RangeSliceCommand.executeLocally(RangeSliceCommand.java:137)
> 	at org.apache.cassandra.service.StorageProxy$LocalRangeSliceRunnable.runMayThrow(StorageProxy.java:1381)
> 	at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1923)
> 	... 3 more
> Caused by: java.io.FileNotFoundException: /var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-36-Data.db
(No such file or directory)
> 	at java.io.RandomAccessFile.open(Native Method)
> 	at java.io.RandomAccessFile.<init>(RandomAccessFile.java:241)
> 	at org.apache.cassandra.io.util.RandomAccessReader.<init>(RandomAccessReader.java:58)
> 	at org.apache.cassandra.io.compress.CompressedRandomAccessReader.<init>(CompressedRandomAccessReader.java:76)
> 	at org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:55)
> 	... 12 more
>  INFO [CompactionExecutor:708] 2014-05-27 17:21:42,810 ColumnFamilyStore.java (line 787)
Enqueuing flush of Memtable-compactions_in_progress@46337516(0/0 serialized/live bytes, 1
ops)
>  WARN [MemoryMeter:1] 2014-05-27 17:21:42,810 Memtable.java (line 470) setting live ratio
to maximum of 64.0 instead of Infinity
>  INFO [MemoryMeter:1] 2014-05-27 17:21:42,811 Memtable.java (line 481) CFS(Keyspace='system',
ColumnFamily='compactions_in_progress') liveRatio is 64.0 (just-counted was 64.0).  calculation
took 0ms for 0 cells
>  INFO [FlushWriter:92] 2014-05-27 17:21:42,811 Memtable.java (line 363) Writing Memtable-compactions_in_progress@46337516(0/0
serialized/live bytes, 1 ops)
>  INFO [FlushWriter:92] 2014-05-27 17:21:42,817 Memtable.java (line 403) Completed flushing
/var/lib/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-jb-51-Data.db
(42 bytes) for commitlog position ReplayPosition(segmentId=1400882649502, position=5391795)
>  INFO [CompactionExecutor:708] 2014-05-27 17:21:42,819 CompactionTask.java (line 275)
Compacted 4 sstables to [/var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-87,].
 56,962,552 bytes to 56,961,531 (~99% of original) in 77,261ms = 0.703107MB/s.  5 total partitions
merged to 2.  Partition merge counts were {1:1, 4:1, }
>  INFO [CompactionExecutor:708] 2014-05-27 17:21:42,820 ColumnFamilyStore.java (line 787)
Enqueuing flush of Memtable-compactions_in_progress@1777281623(179/3463 serialized/live bytes,
9 ops)
>  INFO [MemoryMeter:1] 2014-05-27 17:21:42,820 Memtable.java (line 481) CFS(Keyspace='system',
ColumnFamily='compactions_in_progress') liveRatio is 17.862604748603353 (just-counted was
16.374301675977655).  calculation took 0ms for 7 cells
>  INFO [FlushWriter:92] 2014-05-27 17:21:42,820 Memtable.java (line 363) Writing Memtable-compactions_in_progress@1777281623(179/3197
serialized/live bytes, 9 ops)
>  INFO [FlushWriter:92] 2014-05-27 17:21:42,829 Memtable.java (line 403) Completed flushing
/var/lib/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-jb-52-Data.db
(168 bytes) for commitlog position ReplayPosition(segmentId=1400882649502, position=5392718)
>  INFO [CompactionExecutor:706] 2014-05-27 17:21:42,829 CompactionTask.java (line 115)
Compacting [SSTableReader(path='/var/lib/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-jb-51-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-jb-50-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-jb-52-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-jb-49-Data.db')]
>  INFO [CompactionExecutor:708] 2014-05-27 17:21:42,830 CompactionTask.java (line 115)
Compacting [SSTableReader(path='/var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-71-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-81-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-76-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-87-Data.db')]
>  INFO [CompactionExecutor:708] 2014-05-27 17:21:42,892 CompactionController.java (line
192) Compacting large row weathercql/historical:SFJ (527238028 bytes) incrementally
>  INFO [CompactionExecutor:706] 2014-05-27 17:21:42,894 CompactionTask.java (line 275)
Compacted 4 sstables to [/var/lib/cassandra/data/system/compactions_in_progress/system-compactions_in_progress-jb-53,].
 607 bytes to 321 (~52% of original) in 64ms = 0.004783MB/s.  8 total partitions merged to
4.  Partition merge counts were {1:4, 2:2, }
>  INFO [CompactionExecutor:710] 2014-05-27 17:21:57,469 CompactionController.java (line
192) Compacting large row weathercql/historical:UAK (205611614 bytes) incrementally
> ERROR [ReadStage:61106] 2014-05-27 17:22:20,563 CassandraDaemon.java (line 198) Exception
in thread Thread[ReadStage:61106,5,main]
> java.lang.RuntimeException: java.lang.RuntimeException: java.io.FileNotFoundException:
/var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-36-Data.db (No such
file or directory)
> 	at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1927)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:724)
> Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: /var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-36-Data.db
(No such file or directory)
> 	at org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:59)
> 	at org.apache.cassandra.io.sstable.SSTableReader.openDataReader(SSTableReader.java:1362)
> 	at org.apache.cassandra.io.sstable.SSTableScanner.<init>(SSTableScanner.java:67)
> 	at org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1147)
> 	at org.apache.cassandra.db.RowIteratorFactory.getIterator(RowIteratorFactory.java:69)
> 	at org.apache.cassandra.db.ColumnFamilyStore.getSequentialIterator(ColumnFamilyStore.java:1579)
> 	at org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1699)
> 	at org.apache.cassandra.db.RangeSliceCommand.executeLocally(RangeSliceCommand.java:137)
> 	at org.apache.cassandra.service.StorageProxy$LocalRangeSliceRunnable.runMayThrow(StorageProxy.java:1381)
> 	at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1923)
> 	... 3 more
> Caused by: java.io.FileNotFoundException: /var/lib/cassandra/data/weathercql/historical/weathercql-historical-jb-36-Data.db
(No such file or directory)
> 	at java.io.RandomAccessFile.open(Native Method)
> 	at java.io.RandomAccessFile.<init>(RandomAccessFile.java:241)
> 	at org.apache.cassandra.io.util.RandomAccessReader.<init>(RandomAccessReader.java:58)
> 	at org.apache.cassandra.io.compress.CompressedRandomAccessReader.<init>(CompressedRandomAccessReader.java:76)
> 	at org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:55)
> 	... 12 more
>  INFO [CompactionExecutor:707] 2014-05-27 17:22:20,661 CompactionController.java (line
192) Compacting large row weathercql/historical:MAG (477808828 bytes) incrementally
>  INFO [CompactionExecutor:710] 2014-05-27 17:22:34,154 CompactionController.java (line
192) Compacting large row weathercql/historical:MAG (537030028 bytes) incrementally
> {CODE}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message