cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Kjellman (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-4813) Problem using BulkOutputFormat while streaming several SSTables simultaneously from a given node.
Date Thu, 01 Nov 2012 15:43:15 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-4813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13488774#comment-13488774
] 

Michael Kjellman commented on CASSANDRA-4813:
---------------------------------------------

[~yukim] not sure if this will be an issue for "normal" upgraders or if this was because i
went between patch versions but just wanted you to be aware of the EOF i saw on startup. Node
only has the system schema at this point.

WARN 08:40:29,821 error reading saved cache /ssd/saved_caches/system-local-KeyCache-b.db
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:392)
	at org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:349)
	at org.apache.cassandra.service.CacheService$KeyCacheSerializer.deserialize(CacheService.java:378)
	at org.apache.cassandra.cache.AutoSavingCache.loadSaved(AutoSavingCache.java:144)
	at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:278)
	at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:393)
	at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:365)
	at org.apache.cassandra.db.Table.initCf(Table.java:334)
	at org.apache.cassandra.db.Table.<init>(Table.java:272)
	at org.apache.cassandra.db.Table.open(Table.java:102)
	at org.apache.cassandra.db.Table.open(Table.java:80)
	at org.apache.cassandra.db.SystemTable.checkHealth(SystemTable.java:320)
	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:203)
	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:395)
	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:438)
                
> Problem using BulkOutputFormat while streaming several SSTables simultaneously from a
given node.
> -------------------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-4813
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4813
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 1.1.0
>         Environment: I am using SLES 10 SP3, Java 6, 4 Cassandra + Hadoop nodes, 3 Hadoop
only nodes (datanodes/tasktrackers), 1 namenode/jobtracker. The machines used are Six-Core
AMD Opteron(tm) Processor 8431, 24 cores and 33 GB of RAM. I get the issue on both cassandra
1.1.3, 1.1.5 and I am using Hadoop 0.20.2.
>            Reporter: Ralph Romanos
>            Assignee: Yuki Morishita
>            Priority: Minor
>              Labels: Bulkoutputformat, Hadoop, SSTables
>             Fix For: 1.2.0
>
>         Attachments: 4813.txt
>
>
> The issue occurs when streaming simultaneously SSTables from the same node to a cassandra
cluster using SSTableloader. It seems to me that Cassandra cannot handle receiving simultaneously
SSTables from the same node. However, when it receives simultaneously SSTables from two different
nodes, everything works fine. As a consequence, when using BulkOutputFormat to generate SSTables
and stream them to a cassandra cluster, I cannot use more than one reducer per node otherwise
I get a java.io.EOFException in the tasktracker's logs and a java.io.IOException: Broken pipe
in the Cassandra logs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message