incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Stein <crypt...@gmail.com>
Subject Re: Counter Experience (Performance)?
Date Thu, 27 Oct 2011 16:12:46 GMT
Thanks Jake, bottleneck is the disk I believe each write is taking 50ms, EBS
probably (doing testing in ec2).

I will move my testing over to our production network and run it on some
nodes on some real hardware since that where it will end up.

I am seeing things slow down linearly and nothing dropping
off precipitously.  Glad to have the benchmarks I have good to compare
things.  Thanks!

On Thu, Oct 27, 2011 at 11:30 AM, Jake Luciani <jakers@gmail.com> wrote:

> What's your bottleneck?
> http://spyced.blogspot.com/2010/01/linux-performance-basics.html
>
>
> On Thu, Oct 27, 2011 at 9:37 AM, Joe Stein <cryptcom@gmail.com> wrote:
>
>> Hey folks, I am interested in what others have seen in regards to their
>> experience in the amount of depth and width (CF, Rows & Columns) that they
>> can/do write per batch and simultaneously and what is the inflection point
>> where performance degrades.   I have been expanding my use of counters and
>> am finding some interesting nuances some in my code and implementation
>> related but others I can't yet quantify.
>>
>> My batches are 1x5x5 (1 row for each of 5 column families and 5 columns
>> for each of those 1 rows within each of the 5 column families).  I have 3
>> nodes each with 100 connections and another thread pool of 100 threads
>> rolling through 6,000,000 rows off data sending data out to Cassandra (the
>> 1x5x5 matrice is constructed from each line).  I am finding this to be my
>> sweet spot right now but still not really performing fantastically (or at
>> least what I had hoped) and I am wondering what else (if anything) I can be
>> doing to tweak settings or what to be able to push in more columns or rows.
>>   I find changing my pool settings very much froms this causes error on
>> client lib but I will send email to that list separately though I think I
>> have that figured out on my own for now.
>>
>> Thanks in advance!!!  I hope to get more work going on this in the next
>> day or so in a more methodic way to find the right count so I can build a
>> sparse matrice that will perform best for system and business.
>>
>> /*
>> Joe Stein
>> http://www.linkedin.com/in/charmalloc
>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>> */
>>
>
>
>
> --
> http://twitter.com/tjake
>



-- 

/*
Joe Stein
http://www.linkedin.com/in/charmalloc
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
*/

Mime
View raw message