cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jens Rantil (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-11583) Exception when streaming sstables using `sstableloader`
Date Fri, 15 Apr 2016 09:29:25 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-11583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242689#comment-15242689
] 

Jens Rantil commented on CASSANDRA-11583:
-----------------------------------------

For the record, this is the assertion that fails: https://github.com/apache/cassandra/blob/cassandra-2.1.13/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java#L208

> Exception when streaming sstables using `sstableloader`
> -------------------------------------------------------
>
>                 Key: CASSANDRA-11583
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-11583
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Tools
>         Environment: $ uname -a
> Linux bigdb-100 3.2.0-99-virtual #139-Ubuntu SMP Mon Feb 1 23:52:21 UTC 2016 x86_64 x86_64
x86_64 GNU/Linux
> I am using Datastax Enterprise 4.7.8-1 which is based on 2.1.13.
>            Reporter: Jens Rantil
>
> This bug came out of CASSANDRA-11562.
> I have have a keyspace snapshotted from a 2.1.11 (DSE 4.7.5-1) node. When I'm running
the `sstableloader` I get the following output/exception:
> {noformat}
> # sstableloader --nodes X.X.X.20 --username YYY --password ZZZ --ignore XXX /var/lib/cassandra/data/XXX/ZZZ-f7ebdf0daa3a3062828fddebc109a3b2
> Established connection to initial hosts
> Opening sstables and calculating sections to stream
> Streaming relevant part of /var/lib/cassandra/data/XXX/ZZZ-f7ebdf0daa3a3062828fddebc109a3b2/XXX-ZZZ-ka-6463-Data.db
/var/lib/cassandra/data/XXX/ZZZ-f7ebdf0daa3a3062828fddebc109a3b2/tink-ZZZ-ka-6464-Data.db
to [/X.X.X.33, /X.X.X.113, /X.X.X.32, /X.X.X.20, /X.X.X.122, /X.X.X.176, /X.X.X.143, /X.X.X.172,
/X.X.X.50, /X.X.X.51, /X.X.X.52, /X.X.X.71, /X.X.X.53, /X.X.X.54, /X.X.X.47, /X.X.X.31, /X.X.X.8]
> progress: [/X.X.X.113]0:0/2 0  % [/X.X.X.143]0:0/2 0  % [/X.X.X.172]0:0/2 0  % [/X.X.X.20]0:0/2
0  % [/X.X.X.71]0:0/2 0  % [/X.X.X.122]0:0/2 0  % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2
0  % [/X.X.X.143]0:0/2 0  % [/X.X.X.172]0:0/2 0  % [/X.X.X.20]0:1/2 1  % [/X.X.X.71]0:0/2
0  % [/X.X.X.122]0:0/2 0  % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 0  % [/X.X.X.143]0:0/2
1  % [/X.X.X.172]0:0/2 0  % [/X.X.X.20]0:1/2 1  % [/X.X.X.71]0:0/2 0  % [/X.X.X.122]0:0/2
0  % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 0  % [/X.X.X.143]0:1/2 1  % [/X.X.X.172]0:0/2
0  % [/X.X.X.20]0:1/2 1  % [/X.X.X.71]0:0/2 0  % [/X.X.X.122]0:0/2 0  % [/X.X.X.47]0:0/2 progress:
[/X.X.X.113]0:0/2 0  % [/X.X.X.143]0:1/2 1  % [/X.X.X.172]0:0/2 0  % [/X.X.X.20]0:1/2 1  %
[/X.X.X.71]0:1/2 1  % [/X.X.X.122]0:0/2 0  % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2
0  % [/X.X.X.143]0:1/2 1  % [/X.X.X.172]0:0/2 0  % [/X.X.X.20]0:1/2 1  % [/X.X.X.71]0:1/2
1  % [/X.X.X.122]0:1/2 1  % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 0  % [/X.X.X.143]0:1/2
1  % [/X.X.X.172]0:0/2 0  % [/X.X.X.20]0:1/2 1  % [/X.X.X.71]0:1/2 1  % [/X.X.X.122]0:1/2
1  % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 7  % [/X.X.X.143]0:1/2 1  % [/X.X.X.172]0:0/2
0  % [/X.X.X.20]0:1/2 1  % [/X.X.X.71]0:1/2 1  % [/X.X.X.122]0:1/2 1  % [/X.X.X.47]0:0/2 progress:
[/X.X.X.113]0:0/2 7  % [/X.X.X.143]0:1/2 6  % [/X.X.X.172]0:0/2 0  % [/X.X.X.20]0:1/2 1  %
[/X.X.X.71]0:1/2 1  % [/X.X.X.122]0:1/2 1  % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2
12 % [/X.X.X.143]0:1/2 6  % [/X.X.X.172]0:0/2 0  % [/X.X.X.20]0:1/2 1  % [/X.X.X.71]0:1/2
1  % [/X.X.X.122]0:1/2 1  % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 12 % [/X.X.X.143]0:1/2
11 % [/X.X.X.172]0:0/2 0  % [/X.X.X.20]0:1/2 1  % [/X.X.X.71]0:1/2 1  % [/X.X.X.122]0:1/2
1  % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 19 % [/X.X.X.143]0:1/2 11 % [/X.X.X.172]0:0/2
0  % [/X.X.X.20]0:1/2 1  % [/X.X.X.71]0:1/2 1  % [/X.X.X.122]0:1/2 1  % [/X.X.X.47]0:0/2 progress:
[/X.X.X.113]0:0/2 19 % [/X.X.X.143]0:1/2 15 % [/X.X.X.172]0:0/2 0  % [/X.X.X.20]0:1/2 1  %
[/X.X.X.71]0:1/2 1  % [/X.X.X.122]0:1/2 1  % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2
26 % [/X.X.X.143]0:1/2 15 % [/X.X.X.172]0:0/2 0  % [/X.X.X.20]0:1/2 1  % [/X.X.X.71]0:1/2
1  % [/X.X.X.122]0:1/2 1  % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 26 % [/X.X.X.143]0:1/2
20 % [/X.X.X.172]0:0/2 0  % [/X.X.X.20]0:1/2 1  % [/X.X.X.71]0:1/2 1  % [/X.X.X.122]0:1/2
1  % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2 26 % [/X.X.X.143]0:1/2 21 % [/X.X.X.172]0:0/2
0  % [/X.X.X.20]0:1/2 1  % [/X.X.X.71]0:1/2 1  % [/X.X.X.122]0:1/2 1  % [/X.X.X.47]0:0/2 progress:
[/X.X.X.113]0:0/2 26 % [/X.X.X.143]0:1/2 21 % [/X.X.X.172]0:0/2 0  % [/X.X.X.20]0:1/2 3  %
[/X.X.X.71]0:1/2 1  % [/X.X.X.122]0:1/2 1  % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:0/2
42 % [/X.X.X.143]0:1/2 27 % [/X.X.X.172]0:0/2 0  % [/X.X.X.20]0:1/2 3  % [/X.X.X.71]0:1/2
6  % [/X.X.X.122]0:1/2 1  % [/X.X.X.47]0:0/2 
> [...]
> progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:0/2 78 % [/X.X.X.20]0:2/2
100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:1/2 97 % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:2/2
100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:0/2 78 % [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2
100% [/X.X.X.122]0:1/2 97 % [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2
100% [/X.X.X.172]0:0/2 86 % [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2
100% [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:0/2
86 % [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:0/2 progress:
[/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:0/2 86 % [/X.X.X.20]0:2/2 100%
[/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:2/2
100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:0/2 86 % [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2
100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2
100% [/X.X.X.172]0:0/2 86 % [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2
100% [/X.X.X.47]0:0/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:0/2
95 % [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:0/2 progress:
[/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:0/2 95 % [/X.X.X.20]0:2/2 100%
[/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:1/2 progress: [/X.X.X.113]0:2/2
100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:0/2 95 % [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2
100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2
100% [/X.X.X.172]0:0/2 95 % [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2
100% [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:1/2
98 % [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress:
[/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:1/2 99 % [/X.X.X.20]0:2/2 100%
[/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2
100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:2/2 100% [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2
100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2
100% [/X.X.X.172]0:2/2 100% [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2
100% [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:2/2
100% [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress:
[/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:2/2 100% [/X.X.X.20]0:2/2 100%
[/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2
100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:2/2 100% [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2
100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2
100% [/X.X.X.172]0:2/2 100% [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2
100% [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:2/2
100% [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress:
[/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:2/2 100% [/X.X.X.20]0:2/2 100%
[/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2
100% [/X.X.X.143]0:2/2 100% [/X.X.X.172]0:2/2 100% [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2
100% [/X.X.X.122]0:2/2 100% [/X.X.X.47]0:2/2 progress: [/X.X.X.113]0:2/2 100% [/X.X.X.143]0:2/2
100% [/X.X.X.172]0:2/2 100% [/X.X.X.20]0:2/2 100% [/X.X.X.71]0:2/2 100% [/X.X.X.122]0:2/2
100% [/X.X.X.47]0:2/2 100% [/X.X.X.176]0:2/2 100% [/X.X.X.8]0:2/2 100% total: 100% 0  MB/s(avg:
23 MB/s)
> Summary statistics:
>    Connections per host:         : 1
>    Total files transferred:      : 14
>    Total bytes transferred:      : 142070241
>    Total duration (ms):          : 5823
>    Average transfer rate (MB/s): : 23
>    Peak transfer rate (MB/s):    : 33
> ERROR 09:07:12 [Stream #6e345de0-02e9-11e6-b944-1fee10beb91b] Streaming error occurred
> java.lang.AssertionError: null
>         at org.apache.cassandra.io.sstable.SSTableLoader.releaseReferences(SSTableLoader.java:208)
~[cassandra-all-2.1.13.1218.jar:2.1.13.1218]
>         at org.apache.cassandra.io.sstable.SSTableLoader.onSuccess(SSTableLoader.java:193)
~[cassandra-all-2.1.13.1218.jar:2.1.13.1218]
>         at org.apache.cassandra.io.sstable.SSTableLoader.onSuccess(SSTableLoader.java:48)
~[cassandra-all-2.1.13.1218.jar:2.1.13.1218]
>         at com.google.common.util.concurrent.Futures$4.run(Futures.java:1181) ~[guava-16.0.1.jar:na]
>         at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
~[guava-16.0.1.jar:na]
>         at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
~[guava-16.0.1.jar:na]
>         at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145)
~[guava-16.0.1.jar:na]
>         at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:185)
~[guava-16.0.1.jar:na]
>         at org.apache.cassandra.streaming.StreamResultFuture.maybeComplete(StreamResultFuture.java:213)
~[cassandra-all-2.1.13.1218.jar:2.1.13.1218]
>         at org.apache.cassandra.streaming.StreamResultFuture.handleSessionComplete(StreamResultFuture.java:184)
~[cassandra-all-2.1.13.1218.jar:2.1.13.1218]
>         at org.apache.cassandra.streaming.StreamSession.closeSession(StreamSession.java:415)
~[cassandra-all-2.1.13.1218.jar:2.1.13.1218]
>         at org.apache.cassandra.streaming.StreamSession.complete(StreamSession.java:607)
~[cassandra-all-2.1.13.1218.jar:2.1.13.1218]
>         at org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:471)
~[cassandra-all-2.1.13.1218.jar:2.1.13.1218]
>         at org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:263)
~[cassandra-all-2.1.13.1218.jar:2.1.13.1218]
>         at java.lang.Thread.run(Thread.java:745) [na:1.7.0_80]
> {noformat}
> Question: Is it an issue that I dumped the sstables using a _slightly_ older Cassandra
node (2.1.11 instead of 2.1.13)?
> Let me know if you need anything else from me. Also, know of any possible for workaround?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message