cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Ellis <jbel...@gmail.com>
Subject Re: New User: OSX vs. Debian on Cassandra 0.5.0 with Thrift
Date Fri, 16 Apr 2010 15:38:35 GMT
Sounds like the problem is with the C# client code, then.

On Fri, Apr 16, 2010 at 10:36 AM, Heath Oderman <heath@526valley.com> wrote:
> Ok, it took me a long time to get py_stress working.
> I didn't have thrift / boost / gcc on my debian box :)  ....
> I'm using this command line believing it's similar to my c# tests from a
> remote box:
> cnb:~/apache-cassandra-0.6.0-src/contrib/py_stress# python stress.py -o
> insert -n 1000000 -d 10.113.0.195 -y super
> its wicked fast, as you'd expect.
> total,interval_op_rate,avg_latency,elapsed_time
> 61765,6176,0.00804715166591,10
> 126667,6490,0.00763260502319,20
> 184952,5828,0.00798357854098,30
> 249697,6474,0.00813590502387,40
> 310190,6049,0.00820316256729,50
> 370394,6020,0.00823702464729,60
> 431557,6116,0.00811347469409,70
> 492085,6052,0.00818783322119,80
> 558895,6681,0.00740800772112,90
> 620426,6153,0.00805101232533,100
> 685168,6474,0.00766180823714,110
> 748768,6360,0.00779282277485,120
> 811008,6224,0.00797537056523,130
> 867327,5631,0.00881408287019,140
> This leaves me further stumped.  I guess i will try running py_stress from a
> remote box, because I've got to believe it has something to do with the
> connection.
> Thanks for nudging me toward py_stress.  I'm no closer to understanding, but
> I have more info!
> Stu
> On Thu, Apr 15, 2010 at 1:57 PM, Jonathan Ellis <jbellis@gmail.com> wrote:
>>
>> What kind of numbers do you get from contrib/py_stress?
>>
>> (that's located somewhere else in 0.5, but you should really be using
>> 0.6 anyway.)
>>
>> On Thu, Apr 15, 2010 at 12:53 PM, Heath Oderman <heath@526valley.com>
>> wrote:
>> > So checking it out quickly:
>> > vmstat -
>> > Never swaps.  si and so  stay at 0 during the load.
>> > iostat -x
>> > the %util never climbs above 0.00, but the avgrg-sz jumps bewteen
>> > samples
>> > from 0 - 30 - 90 - 0 (5 second intervals)
>> > top shows the cpu barely working and mem utilization is below 20%.
>> > Still slow.  :(
>> > Thanks for the suggestions.  In your article on your blog it'd be
>> > awesome to
>> > include some implications, like "avgrg-sz over 250 may mean XXX"  Even
>> > if
>> > it's utterly hardware and system dependent it'd give a guy like me an
>> > idea
>> > if what I was seeing was bad or good. :D
>> > Thanks again,
>> > Heath
>> >
>> > On Thu, Apr 15, 2010 at 1:34 PM, Heath Oderman <heath@526valley.com>
>> > wrote:
>> >>
>> >> Thanks Jonathan, I'll check this out right away.
>> >>
>> >> On Thu, Apr 15, 2010 at 1:32 PM, Jonathan Ellis <jbellis@gmail.com>
>> >> wrote:
>> >>>
>> >>> You're right, to get those numbers on debian something is very wrong.
>> >>>
>> >>> Have you looked at
>> >>> http://spyced.blogspot.com/2010/01/linux-performance-basics.html ?
>> >>> What is the bottleneck on the linux machines?
>> >>>
>> >>> With the kind of speed you are seeing I wouldn't be surprised if it
is
>> >>> swapping.
>> >>>
>> >>> -Jonathan
>> >>>
>> >>> On Tue, Apr 13, 2010 at 11:38 PM, Heath Oderman <heath@526valley.com>
>> >>> wrote:
>> >>> > Hi,
>> >>> > I wrote a few days ago and got a few good suggestions.  I'm still
>> >>> > seeing
>> >>> > dramatic differences between Cassandra 0.5.0 on OSX vs. Debian
>> >>> > Linux.
>> >>> > I've tried on Debian with the Sun JRE and the Open JDK with nearly
>> >>> > identical
>> >>> > results. I've tried a mix of hardware.
>> >>> > Attached are some graphs I've produced of my results which show
that
>> >>> > in
>> >>> > OSX,
>> >>> > Cassandra takes longer with a greater load but is wicked fast
>> >>> > (expected).
>> >>> > In the SunJDK or Open JDK on Debian I get amazingly consistent
time
>> >>> > taken to
>> >>> > do the writes, regardless of the load and the times are always
>> >>> > ridiculously
>> >>> > high.  It's insanely slow.
>> >>> > I genuinely believe that I must be doing something very wrong in
my
>> >>> > Debian
>> >>> > setups, but they are all vanilla installs, both 64 bit and 32 bit
>> >>> > machines,
>> >>> > 64bit and 32 bit installs.  Cassandra packs taken from
>> >>> > http://www.apache.org/dist/cassandra/debian.
>> >>> > I am using Thrift, and I'm using a c# client because that's how
I
>> >>> > intend to
>> >>> > actually use Cassandra and it seems pretty sensible.
>> >>> > An example of what I'm seeing is:
>> >>> > 5 Threads Each writing 100,000 Simple Entries
>> >>> > OSX: 1 min 16 seconds ~ 6515 Entries / second
>> >>> > Debian: 1 hour 15 seconds ~ 138 Records / second
>> >>> > 15 Threads Each writing 100,000 Simple Entries
>> >>> > OSX: 2min 30 seconds seconds writing ~10,000 Entries / second
>> >>> > Debian: 1 hour 1.5 minutes ~406 Entries / second
>> >>> > 20 Threads Each Writing 100,000 Simple Entries
>> >>> > OSX: 3min 19 seconds ~ 10,050 Entries / second
>> >>> > Debian: 1 hour 20 seconds ~ 492 Entries / second
>> >>> > If anyone has any suggestions or pointers I'd be glad to hear them.
>> >>> > Thanks,
>> >>> > Stu
>> >>> > Attached:
>> >>> > 1. CassLoadTesting.ods (all my results and graphs in OpenOffice
>> >>> > format
>> >>> > downloaded from Google Docs)
>> >>> > 2. OSX Records per Second - a graph of how many entries get written
>> >>> > per
>> >>> > second for 10,000 & 100,000 entries as thread count is increased
in
>> >>> > OSX.
>> >>> > 3. Open JDK Records per Second - the same graph but of Open JDK
on
>> >>> > Debian
>> >>> > 4. Open JDK Total Time By Thread - the total time taken from test
>> >>> > start
>> >>> > to
>> >>> > finish (all threads completed) to write 10,000 & 100,000 entries
as
>> >>> > thread
>> >>> > count is increased in Debian with Open JDK
>> >>> > 5. OSX Total time by Thread - same as 4, but for OSX.
>> >>> >
>> >>> >
>> >>
>> >
>> >
>
>

Mime
View raw message