cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Henrik Baastrup (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-5987) IllegalArgumentException when bulk loading
Date Tue, 10 Sep 2013 14:58:52 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13763092#comment-13763092
] 

Henrik Baastrup commented on CASSANDRA-5987:
--------------------------------------------

Hi Yuki,

I see your point! I have now simplified my program and table so I do not use composite keys,
but only one primary key; I use a UUID I generate on the fly for this. 
The problem remain, I still get the IllegalArgumentException.

I changed my create table statement as follow:
{quote}
CREATE TABLE nsmo_dns (
  key uuid PRIMARY KEY,
  endtime bigint,
  lastframetime bigint,
  .
  .
{quote}

and my SSTableSimpleUnsortedWriter create statement is as follow:
{quote}
new SSTableSimpleUnsortedWriter(
  directory,
  partitioner,
  keyspace,
  columnFamily,
  UUIDType.instance,
  null,
  BUFFER_SIZE);
{quote}

My write looks like this now:
{quote}
public void write(SSTableSimpleUnsortedWriter writer, long timestamp) throws IOException {
  writer.newRow(ByteBuffer.wrap(decompose(key)));
        
  writer.addColumn(toBufferWithShortLength("endtime"), bytes(endtime), timestamp);
  writer.addColumn(toBufferWithShortLength("lastframetime"), bytes(lastframetime), timestamp);
  writer.addColumn(bytes("begintime"), bytes(begintime), timestamp);
  .
  .
{quote}

As I see it, either does SSTableSimpleUnsortedWriter not write the length of the column name
in front of it, or the AbstractCompositeType in the Cassandra engine expect a length of the
column name where there is none.
Of cause, there exist also the possibility that I have missed something ;)
                
> IllegalArgumentException when bulk loading
> ------------------------------------------
>
>                 Key: CASSANDRA-5987
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5987
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: Cassandra 2.0.0 on Linux
>            Reporter: Henrik Baastrup
>         Attachments: CassandraLoader.tgz
>
>
> I have created SSTables with the org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter
class, but when I try to load the SSTables with "sstableloader -d localhost nsmo_ds/nsmo_ds"
I get:
>  WARN 13:46:40,875 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream failed
> Streaming to the following hosts failed:
> {quote}
> [/127.0.0.1]
> java.util.concurrent.ExecutionException: org.apache.cassandra.streaming.StreamException:
Stream failed
>  WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream failed
> ERROR 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming error occurred
> java.io.IOException: Broken pipe
> 	at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
> 	at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> 	at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> 	at sun.nio.ch.IOUtil.write(Unknown Source)
> 	at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> 	at org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:43)
> 	at org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:384)
> 	at org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:372)
> 	at java.lang.Thread.run(Unknown Source)
>  WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream failed
> {quote}
> in the server log I have:
> {quote}
> ERROR [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,859 StreamSession.java (line 409)
[Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming error occurred
> java.io.IOException: Too many retries for Header (cfId: 976cb58f-d1f4-39c6-b6cc-8aea7a64daca,
#0, version: ja, estimated keys: 62208, transfer size: 54794676, compressed?: false)
> 	at org.apache.cassandra.streaming.StreamSession.doRetry(StreamSession.java:522)
> 	at org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:59)
> 	at org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:45)
> 	at org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:54)
> 	at org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:287)
> 	at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.IllegalArgumentException
> 	at java.nio.Buffer.limit(Unknown Source)
> 	at org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:55)
> 	at org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:64)
> 	at org.apache.cassandra.db.marshal.AbstractCompositeType.split(AbstractCompositeType.java:130)
> 	at org.apache.cassandra.io.sstable.ColumnNameHelper.minComponents(ColumnNameHelper.java:101)
> 	at org.apache.cassandra.io.sstable.SSTableWriter.appendFromStream(SSTableWriter.java:254)
> 	at org.apache.cassandra.streaming.StreamReader.writeRow(StreamReader.java:134)
> 	at org.apache.cassandra.streaming.StreamReader.read(StreamReader.java:88)
> 	at org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:55)
> 	... 4 more
>  INFO [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,862 StreamResultFuture.java (line
180) [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Session with /172.27.14.242 is complete
>  WARN [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,864 StreamResultFuture.java (line
209) [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream fail
> {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message