incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jedd Rashbrooke <>
Subject Re: Dazed and confused with Cassandra on EC2 ...
Date Tue, 28 Sep 2010 17:11:57 GMT
 Peter - my apologies for the slow response - we had
 to divert down a 'Plan B' approach last week involving
 MySQL, memcache, redis and various other uglies.

On 20 September 2010 23:11, Peter Schuller <> wrote:
> Are you running an old JVM by any chance? (Just grasping for straws.)

 JVM is Sun's 1.6 - I've been caught out once before with
 openjdk's performance challenges, so I'm particularly
 careful with this now.

> Hmm. I can see useless spinning decreasing efficiency, but the numbers
> from your log are really extreme. Do you have a URL / bug id or
> anything that one can read up on about this?

 We've rebuilt the Cassandra cluster this week, avoiding
 Hadoop entirely - partly to reduce the variables in play,
 and partly because it looks like we'll only need two 'feeder'
 nodes for our jobs with the size of Cassandra cluster that
 we're likely going to end up with (10-12 ish).  Any ratio
 higher than that seems to, on EC2 at least, cause too many
 fails on the Cassandra side.

 Actually, there's a question - is it 'acceptable' do you think
 for GC to take out a small number of your nodes at a time,
 so long as the bulk (or at least where RF is > nodes gone
 on STW GC) of the nodes are okay?  I suspect this is a
 question peculiar to Amazon EC2, as I've never seen a box
 rendered non-communicative by a single core flat-lining.

 By the end of this week we hope to have a better idea (mind,
 I've thought that for the past 5 weeks of experimenting).  If I'm
 back to square one at that point I'll start pastebining some logs
 and configs.  Increasingly, I'm convinced that many of these
 problems would be solved if we hosted our own servers.


View raw message