incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benoit Perroud <ben...@noisette.ch>
Subject Bulk loading errors with 1.0.8
Date Thu, 05 Apr 2012 15:58:00 GMT
Hi All,

I'm experiencing the following errors while bulk loading data into a cluster

ERROR [Thread-23] 2012-04-05 09:58:12,252 AbstractCassandraDaemon.java
(line 139) Fatal exception in thread Thread[Thread-23,5,main]
java.lang.RuntimeException: Insufficient disk space to flush
7813594056494754913 bytes
        at org.apache.cassandra.db.ColumnFamilyStore.getFlushPath(ColumnFamilyStore.java:635)
        at org.apache.cassandra.streaming.StreamIn.getContextMapping(StreamIn.java:92)
        at org.apache.cassandra.streaming.IncomingStreamReader.<init>(IncomingStreamReader.java:68)
        at org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:185)
        at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:81)

Here I'm not really sure I was able to generate 7 exa bytes of data ;)


ERROR [Thread-46] 2012-04-05 09:58:14,453 AbstractCassandraDaemon.java
(line 139) Fatal exception in thread Thread[Thread-46,5,main]
java.lang.NullPointerException
        at org.apache.cassandra.io.sstable.SSTable.getMinimalKey(SSTable.java:156)
        at org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:334)
        at org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:302)
        at org.apache.cassandra.streaming.IncomingStreamReader.streamIn(IncomingStreamReader.java:155)
        at org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:89)
        at org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:185)
        at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:81)

This one sounds like a null key added to the SSTable at some point,
but I'm rather confident I'm checking for key nullity.

Errors are seen in different nodes. Some nodes succeeded. All 5
cluster nodes run 1.0.8 with JNA enabled.

Basically I'm generating SSTables in a Hadoop Reducer, storing them in
HDFS. After the job finished, I download them back into a single node
and streaming the files into Cassandra (Yes, I definitely need to try
1.1).

Do someone have a hint or a pointer where to start looking ?

Thanks,

Benoit.

Mime
View raw message