cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Ellis (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (CASSANDRA-6171) Bulk loading to another keyspace with sstableloader throws EOFException
Date Wed, 09 Oct 2013 13:22:43 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-6171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Ellis resolved CASSANDRA-6171.
---------------------------------------

    Resolution: Not A Problem

The design is not very friendly, but it is working as designed.

This will be improved by CASSANDRA-5202 moving the KS name out of the filename.

> Bulk loading to another keyspace with sstableloader throws EOFException
> -----------------------------------------------------------------------
>
>                 Key: CASSANDRA-6171
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6171
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Tools
>         Environment: Linux Mint x64 (Olive)
> Cassandra 1.2.9
> [cqlsh 3.1.7 | Cassandra 1.2.9-1~precise~ppa1 | CQL spec 3.0.0 | Thrift protocol 19.36.0]
>            Reporter: palacsint
>              Labels: sstableloader
>
> I'm trying to to load a snapshot of keyspace {{test}} to another keyspace, called {{test2}}.
> I've got the {{java.io.EOFException}}s below.
> *Steps to reproduce*:
> {noformat}
> # cqlsh
> cqlsh> CREATE KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 'replication_factor':
1};
> cqlsh> CREATE TABLE test.test (key varchar PRIMARY KEY, value varchar);
> cqlsh> INSERT INTO test.test (key, value) VALUES ('ryan', 'ryan');
> # nodetool --host localhost snapshot
> Requested creating snapshot for: all keyspaces 
> Snapshot directory: 1381309132532
> # mkdir test2
> # mkdir test2/test
> # cp -R /var/lib/cassandra/data/test/test/snapshots/1381309132532/* test2/test/
> # cqlsh
> cqlsh> CREATE KEYSPACE test2 WITH replication = {'class': 'SimpleStrategy', 'replication_factor':
1};
> cqlsh> CREATE TABLE test2.test (key varchar PRIMARY KEY, value varchar);
> cqlsh> 
> # sstableloader --verbose --debug --nodes localhost test2/test/
> Streaming revelant part of test2/test/test-test-ic-1-Data.db  to [/127.0.0.1]
> progress: [/127.0.0.1 0/1 (0)] [total: 0 - 0MB/s (avg: 0MB/s)]Streaming session to /127.0.0.1
failed
> ERROR 11:02:43,612 Error in ThreadPoolExecutor
> java.lang.RuntimeException: java.io.EOFException
> 	at com.google.common.base.Throwables.propagate(Throwables.java:160)
> 	at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:724)
> Caused by: java.io.EOFException
> 	at java.io.DataInputStream.readInt(DataInputStream.java:392)
> 	at org.apache.cassandra.streaming.FileStreamTask.receiveReply(FileStreamTask.java:193)
> 	at org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:101)
> 	at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> 	... 3 more
> Exception in thread "Streaming to /127.0.0.1:1" java.lang.RuntimeException: java.io.EOFException
> 	at com.google.common.base.Throwables.propagate(Throwables.java:160)
> 	at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:724)
> Caused by: java.io.EOFException
> 	at java.io.DataInputStream.readInt(DataInputStream.java:392)
> 	at org.apache.cassandra.streaming.FileStreamTask.receiveReply(FileStreamTask.java:193)
> 	at org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:101)
> 	at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> 	... 3 more
> progress: [/127.0.0.1 1/1 (100)] [total: 100 - 0MB/s (avg: 0MB/s)]Streaming to the following
hosts failed:
> [/127.0.0.1]
> # 
> {noformat}
> The {{system.log}} file contains the following:
> {noformat}
> ERROR [Thread-4] 2013-10-09 11:02:43,608 CassandraDaemon.java (line 192) Exception in
thread Thread[Thread-4,5,main]
> java.lang.AssertionError
> 	at org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:168)
> 	at org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:138)
> 	at org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:238)
> 	at org.apache.cassandra.net.IncomingTcpConnection.handleStream(IncomingTcpConnection.java:178)
> 	at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:78)
> {noformat}
> *A workaround is renaming the snapshot files too* (not just the directory):
> {noformat}
> # rename --verbose 's/test-test/test2-test/' test2/test/*
> test2/test/test-test-ic-1-CompressionInfo.db renamed as test2/test/test2-test-ic-1-CompressionInfo.db
> test2/test/test-test-ic-1-Data.db renamed as test2/test/test2-test-ic-1-Data.db
> test2/test/test-test-ic-1-Filter.db renamed as test2/test/test2-test-ic-1-Filter.db
> test2/test/test-test-ic-1-Index.db renamed as test2/test/test2-test-ic-1-Index.db
> test2/test/test-test-ic-1-Statistics.db renamed as test2/test/test2-test-ic-1-Statistics.db
> test2/test/test-test-ic-1-Summary.db renamed as test2/test/test2-test-ic-1-Summary.db
> test2/test/test-test-ic-1-TOC.txt renamed as test2/test/test2-test-ic-1-TOC.txt
> # sstableloader --verbose --debug --nodes localhost test2/test/
> Streaming revelant part of test2/test/test2-test-ic-1-Data.db  to [/127.0.0.1]
> progress: [/127.0.0.1 0/1 (0)] [total: 0 - 0MB/s (avg: 0MB/s)]Streaming session to /127.0.0.1
completed (waiting on 0 outstanding sessions)
> progress: [/127.0.0.1 1/1 (100)] [total: 100 - 0MB/s (avg: 0MB/s)]
> # 
> {noformat}
> {{sstableloader --help}} mentions this:
> {noformat}
> sstableloader  --help
> usage: sstableloader [options] <dir_path>
> --
> Bulk load the sstables found in the directory <dir_path> to the configured
> cluster.The parent directory of <dir_path> is used as the keyspace name.
> So for instance, to load an sstable named Standard1-g-1-Data.db into
> keyspace Keyspace1, you will need to have the files Standard1-g-1-Data.db
> and Standard1-g-1-Index.db in a directory Keyspace1/Standard1/ in the
> directory and call: sstableloader Keyspace1/Standard1
> {noformat}
> However, renaming the snapshot files to the suggested  format (if I'm right here) still
not help:
> {noformat}
> # rename --verbose 's/test2-test-/test-/' test2/test/*
> test2/test/test2-test-ic-1-CompressionInfo.db renamed as test2/test/test-ic-1-CompressionInfo.db
> test2/test/test2-test-ic-1-Data.db renamed as test2/test/test-ic-1-Data.db
> test2/test/test2-test-ic-1-Filter.db renamed as test2/test/test-ic-1-Filter.db
> test2/test/test2-test-ic-1-Index.db renamed as test2/test/test-ic-1-Index.db
> test2/test/test2-test-ic-1-Statistics.db renamed as test2/test/test-ic-1-Statistics.db
> test2/test/test2-test-ic-1-Summary.db renamed as test2/test/test-ic-1-Summary.db
> test2/test/test2-test-ic-1-TOC.txt renamed as test2/test/test-ic-1-TOC.txt
> # sstableloader --verbose --debug --nodes localhost test2/test/
> Skipping file test-ic-1-Data.db: column family test2.ic doesn't exist
> No sstables to stream
> progress: [total: 100 - 0MB/s (avg: 0MB/s)][11:42:53] (0)
> {noformat}
> An upgraded {{sstableloader --help}} message and a helpful error message (instead of
the {{EOFException}}) would be much better here.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message