incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Julie <>
Subject Re: java.lang.RuntimeException: Value too large for defined data type
Date Tue, 15 Jun 2010 22:15:15 GMT
Phil Stanhope <pstanhope <at>> writes:

> How are you doing your inserts?
> I draw a clear line between 1) bootstrapping a cluster with data and 2)
simulating expected/projected
> read/write behavior.
> If you are bootstrapping then I would look into the batch_mutate APIs. They
allow you to improve your
> performance on writes dramatically.
> If you are read/write testing on a populated cluster, insert and batch_insert
(for super columns) are the
> way to go.
> As Ben has pointed to me in numerous threads ... think carefully about your
replication factor. Do you want
> the data on all nodes? Or sufficiently replicated so that you can recover? Do
you want consistency at the
> time of write? Or eventually?
> Cassandra has a bunch of knobs that you can turn ... but that flexibility
requires that you think about your
> expected usage patterns and operational policies.
> -phil

My inserts are being done 100 rows at a time using batch_mutate().
I bring up all 10 nodes in my cassandra cluster at once (no live bootstrapping 
of nodes).  Once they are up, I begin populating the database running 8 write 
clients (on 8 different VMs), each writing 100 rows at a time.  As mentioned 
earlier, each client writes to a different cassandra server node so no one 
server node is fielding all the writes simultaneously.

I have a replication factor of 3 because I need to be able to survive 2 out of 
10 nodes going down at once.

I am baffled by all the "Value too large" exceptions that are occurring on 
every one of my 10 servers:
 ERROR [MESSAGE-STREAMING-POOL:1] 2010-06-14 19:30:24,471 (line 101) Error in ThreadPoolExecutor
java.lang.RuntimeException: Value too large for defined
data type
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask
        at java.util.concurrent.ThreadPoolExecutor$

It seems to be happening just after this is logged:
INFO [AE-SERVICE-STAGE:1]  2010-06-14 19:28:39,851

I'm also baffled that after all compactions are done on every one of the 10 
servers, about 5 out of 10 servers are still at 40% CPU usage, although they 
are doing 0 disk IO. I am not running anything else running on these server 
nodes except for cassandra.  The compactions have been done for over an hour.
The last write took place 5 hours ago.

Thank you for any help,

View raw message