cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sonny Heer <sonnyh...@gmail.com>
Subject Bulk Ingestion Issues
Date Wed, 24 Feb 2010 17:30:13 GMT
I have a single box, and trying to ingest some data into a single
keyspace and 5 CFs.  Basically it reads from a directory text files,
and inserts into Cassandra.  I've set the BinaryMemtableSizeInMB to
64. For some reason I'm not getting all my data into cassandra.  I get
some ingested, but very little.   Is this because I'm only using a
single box, and it cant handle the load? There is an exception when
the ingest is about to finish. here is the output from a clean startup
to end:


:~/apache-cassandra-incubating-0.5.0$ bin/cassandra -f
Listening for transport dt_socket at address: 8888
INFO - Saved Token not found. Using eGsC7VsC6xz0uskJ
INFO - Starting up server gossip
INFO - Cassandra starting up...
INFO - Node /127.0.0.1 is now part of the cluster
INFO - InetAddress /127.0.0.1 is now UP
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@17b650a
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@17b650a
INFO - Writing org.apache.cassandra.db.BinaryMemtable@17b650a
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@ec44cb
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@ec44cb
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily4-1-Data.db
INFO - Writing org.apache.cassandra.db.BinaryMemtable@ec44cb
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily3-1-Data.db
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@b11287
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@b11287
INFO - Writing org.apache.cassandra.db.BinaryMemtable@b11287
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily2-1-Data.db
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@1687dcd
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@1687dcd
INFO - Writing org.apache.cassandra.db.BinaryMemtable@1687dcd
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily1-1-Data.db
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@137bc9
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@137bc9
INFO - Writing org.apache.cassandra.db.BinaryMemtable@137bc9
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily4-2-Data.db
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@1d4f6b4
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@1d4f6b4
INFO - Writing org.apache.cassandra.db.BinaryMemtable@1d4f6b4
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily3-2-Data.db
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@1ac9cff
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@1ac9cff
INFO - Writing org.apache.cassandra.db.BinaryMemtable@1ac9cff
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@11c9fcc
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@11c9fcc
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily5-1-Data.db
INFO - Writing org.apache.cassandra.db.BinaryMemtable@11c9fcc
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily2-2-Data.db
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@16a90c9
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@16a90c9
INFO - Writing org.apache.cassandra.db.BinaryMemtable@16a90c9
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily1-2-Data.db
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@118bd3c
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@118bd3c
INFO - Writing org.apache.cassandra.db.BinaryMemtable@118bd3c
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily4-3-Data.db
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@7a9bff
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@7a9bff
INFO - Writing org.apache.cassandra.db.BinaryMemtable@7a9bff
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily3-3-Data.db
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@106bcba
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@106bcba
INFO - Writing org.apache.cassandra.db.BinaryMemtable@106bcba
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily2-3-Data.db
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@1ad552c
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@1ad552c
INFO - Writing org.apache.cassandra.db.BinaryMemtable@1ad552c
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily1-3-Data.db
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@272111
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@272111
INFO - Writing org.apache.cassandra.db.BinaryMemtable@272111
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily4-4-Data.db
INFO - Compacting
[org.apache.cassandra.io.SSTableReader(path='/var/lib/cassandra/data/Keyspace1/ColumnFamily4-1-Data.db'),org.apache.cassandra.io.SSTableReader(path='/var/lib/cassandra/data/Keyspace1/ColumnFamily4-2-Data.db'),org.apache.cassandra.io.SSTableReader(path='/var/lib/cassandra/data/Keyspace1/ColumnFamily4-3-Data.db'),org.apache.cassandra.io.SSTableReader(path='/var/lib/cassandra/data/Keyspace1/ColumnFamily4-4-Data.db')]
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@de7bf0
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@de7bf0
INFO - Writing org.apache.cassandra.db.BinaryMemtable@de7bf0
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily3-4-Data.db
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@19db5ef
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@19db5ef
INFO - Writing org.apache.cassandra.db.BinaryMemtable@19db5ef
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@2963dd
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@2963dd
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily5-2-Data.db
INFO - Writing org.apache.cassandra.db.BinaryMemtable@2963dd
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily2-4-Data.db
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@e60bff
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@e60bff
INFO - Writing org.apache.cassandra.db.BinaryMemtable@e60bff
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily1-4-Data.db
INFO - Sending AEService tree for Pair(left=Keyspace1,
right=ColumnFamily4) to: []
INFO - Compacted to
/var/lib/cassandra/data/Keyspace1/ColumnFamily4-5-Data.db.
130485403/127114680 bytes for 611764 keys.  Time: 162272ms.
INFO - Compacting
[org.apache.cassandra.io.SSTableReader(path='/var/lib/cassandra/data/Keyspace1/ColumnFamily3-1-Data.db'),org.apache.cassandra.io.SSTableReader(path='/var/lib/cassandra/data/Keyspace1/ColumnFamily3-2-Data.db'),org.apache.cassandra.io.SSTableReader(path='/var/lib/cassandra/data/Keyspace1/ColumnFamily3-3-Data.db'),org.apache.cassandra.io.SSTableReader(path='/var/lib/cassandra/data/Keyspace1/ColumnFamily3-4-Data.db')]
INFO - Deleted /var/lib/cassandra/data/Keyspace1/ColumnFamily4-2-Data.db
INFO - Deleted /var/lib/cassandra/data/Keyspace1/ColumnFamily4-3-Data.db
INFO - Deleted /var/lib/cassandra/data/Keyspace1/ColumnFamily4-1-Data.db
INFO - Deleted /var/lib/cassandra/data/Keyspace1/ColumnFamily4-4-Data.db
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@162e7ac
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@162e7ac
INFO - Writing org.apache.cassandra.db.BinaryMemtable@162e7ac
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily4-6-Data.db
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@500293
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@500293
INFO - Writing org.apache.cassandra.db.BinaryMemtable@500293
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily3-6-Data.db
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@1b83238
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@1b83238
INFO - Writing org.apache.cassandra.db.BinaryMemtable@1b83238
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily2-5-Data.db
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@10ec35f
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@10ec35f
INFO - Writing org.apache.cassandra.db.BinaryMemtable@10ec35f
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily1-5-Data.db
INFO - Sending AEService tree for Pair(left=Keyspace1,
right=ColumnFamily3) to: []
INFO - Compacted to
/var/lib/cassandra/data/Keyspace1/ColumnFamily3-5-Data.db.
128120268/123412906 bytes for 616113 keys.  Time: 163284ms.
INFO - Compacting
[org.apache.cassandra.io.SSTableReader(path='/var/lib/cassandra/data/Keyspace1/ColumnFamily2-1-Data.db'),org.apache.cassandra.io.SSTableReader(path='/var/lib/cassandra/data/Keyspace1/ColumnFamily2-2-Data.db'),org.apache.cassandra.io.SSTableReader(path='/var/lib/cassandra/data/Keyspace1/ColumnFamily2-3-Data.db'),org.apache.cassandra.io.SSTableReader(path='/var/lib/cassandra/data/Keyspace1/ColumnFamily2-4-Data.db'),org.apache.cassandra.io.SSTableReader(path='/var/lib/cassandra/data/Keyspace1/ColumnFamily2-5-Data.db')]
INFO - Deleted /var/lib/cassandra/data/Keyspace1/ColumnFamily3-3-Data.db
INFO - Deleted /var/lib/cassandra/data/Keyspace1/ColumnFamily3-4-Data.db
INFO - Deleted /var/lib/cassandra/data/Keyspace1/ColumnFamily3-1-Data.db
INFO - Deleted /var/lib/cassandra/data/Keyspace1/ColumnFamily3-2-Data.db
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@174fbfe
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@174fbfe
INFO - Writing org.apache.cassandra.db.BinaryMemtable@174fbfe
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily4-7-Data.db
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@1a23aaf
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@1a23aaf
INFO - Writing org.apache.cassandra.db.BinaryMemtable@1a23aaf
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily3-7-Data.db
INFO - Enqueuing flush of org.apache.cassandra.db.BinaryMemtable@15ddd1f
INFO - Sorting org.apache.cassandra.db.BinaryMemtable@15ddd1f
INFO - Writing org.apache.cassandra.db.BinaryMemtable@15ddd1f
WARN - Problem reading from socket connected to :
java.nio.channels.SocketChannel[connected local=/192.168.240.85:7000
remote=/127.0.0.1:59883]
WARN - Exception was generated at : 02/24/2010 09:17:57 on thread
MESSAGING-SERVICE-POOL:4
Reached an EOL or something bizzare occured. Reading from: /127.0.0.1
BufferSizeRemaining: 16
java.io.IOException: Reached an EOL or something bizzare occured.
Reading from: /127.0.0.1 BufferSizeRemaining: 16
        at org.apache.cassandra.net.io.StartState.doRead(StartState.java:44)
        at org.apache.cassandra.net.io.ProtocolState.read(ProtocolState.java:39)
        at org.apache.cassandra.net.io.TcpReader.read(TcpReader.java:95)
        at org.apache.cassandra.net.TcpConnection$ReadWorkItem.run(TcpConnection.java:445)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)

INFO - Closing errored connection
java.nio.channels.SocketChannel[connected local=/192.168.240.85:7000
remote=/127.0.0.1:59883]
INFO - Completed flushing
/var/lib/cassandra/data/Keyspace1/ColumnFamily5-3-Data.db
INFO - Sending AEService tree for Pair(left=Keyspace1,
right=ColumnFamily2) to: []
INFO - InetAddress /127.0.0.1 is now dead.
INFO - Compacted to
/var/lib/cassandra/data/Keyspace1/ColumnFamily2-6-Data.db.
140205416/122637008 bytes for 628374 keys.  Time: 167968ms.
INFO - Compacting
[org.apache.cassandra.io.SSTableReader(path='/var/lib/cassandra/data/Keyspace1/ColumnFamily1-1-Data.db'),org.apache.cassandra.io.SSTableReader(path='/var/lib/cassandra/data/Keyspace1/ColumnFamily1-2-Data.db'),org.apache.cassandra.io.SSTableReader(path='/var/lib/cassandra/data/Keyspace1/ColumnFamily1-3-Data.db'),org.apache.cassandra.io.SSTableReader(path='/var/lib/cassandra/data/Keyspace1/ColumnFamily1-4-Data.db'),org.apache.cassandra.io.SSTableReader(path='/var/lib/cassandra/data/Keyspace1/ColumnFamily1-5-Data.db')]
INFO - Deleted /var/lib/cassandra/data/Keyspace1/ColumnFamily2-5-Data.db
INFO - Deleted /var/lib/cassandra/data/Keyspace1/ColumnFamily2-2-Data.db
INFO - Deleted /var/lib/cassandra/data/Keyspace1/ColumnFamily2-4-Data.db
INFO - Deleted /var/lib/cassandra/data/Keyspace1/ColumnFamily2-1-Data.db
INFO - Deleted /var/lib/cassandra/data/Keyspace1/ColumnFamily2-3-Data.db
INFO - Sending AEService tree for Pair(left=Keyspace1,
right=ColumnFamily1) to: []
INFO - Compacted to
/var/lib/cassandra/data/Keyspace1/ColumnFamily1-6-Data.db.
21961274/11617926 bytes for 55594 keys.  Time: 12443ms.
INFO - Deleted /var/lib/cassandra/data/Keyspace1/ColumnFamily1-4-Data.db
INFO - Deleted /var/lib/cassandra/data/Keyspace1/ColumnFamily1-5-Data.db
INFO - Deleted /var/lib/cassandra/data/Keyspace1/ColumnFamily1-1-Data.db
INFO - Deleted /var/lib/cassandra/data/Keyspace1/ColumnFamily1-3-Data.db
INFO - Deleted /var/lib/cassandra/data/Keyspace1/ColumnFamily1-2-Data.db

Mime
View raw message