Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 69662 invoked from network); 26 Mar 2010 20:08:41 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 26 Mar 2010 20:08:41 -0000 Received: (qmail 63180 invoked by uid 500); 26 Mar 2010 20:08:40 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 63162 invoked by uid 500); 26 Mar 2010 20:08:40 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 63154 invoked by uid 99); 26 Mar 2010 20:08:40 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 26 Mar 2010 20:08:40 +0000 X-ASF-Spam-Status: No, hits=0.9 required=10.0 tests=AWL,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of scottblanc@gmail.com designates 209.85.212.44 as permitted sender) Received: from [209.85.212.44] (HELO mail-vw0-f44.google.com) (209.85.212.44) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 26 Mar 2010 20:08:36 +0000 Received: by vws12 with SMTP id 12so2195531vws.31 for ; Fri, 26 Mar 2010 13:08:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:received:message-id:subject:from:to:content-type; bh=yR7kzslKF0dGXw6ptF1oJ6gXExbEiVeDLsy4mUh+fnc=; b=jPwnn1Sfh1XJXrA1v2D288rLkP5hOqrbOlV5RAnpI+0wtcPDBdDp6+urzysgE5atuS TuS3hjUF03eUNVXz5fTlerhz7aEBJTdnG0z0TTVI8zpqsLgUKlO5CJzVH1mL79oe0ueu HM4Jf2IK78KS7HKhWiiFpoURy2I8u3jUG/hDg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=Wl0BumTG+RSvY1t3pXEjH+3Q9yUlnGzCXYtv+Kqfq2ZRpgHs8oaeCYzDSlFERlbcQr sGUJeMwGyq7enjh4B9tsGohTQbO9KxFZj5K3vZJ7O/82+RDvEfUoDd5Sm4rRgybBptax fo321PBPI22UA1RgTqS57LP3eCO+iblXGYA8U= MIME-Version: 1.0 Received: by 10.220.111.79 with HTTP; Fri, 26 Mar 2010 13:08:15 -0700 (PDT) In-Reply-To: <7c400caa1003261300he3d7a07l81c4d3673a04b1c9@mail.gmail.com> References: <7c400caa1003260845n869f806q5ae4c77c9bf25c07@mail.gmail.com> <7c400caa1003261300he3d7a07l81c4d3673a04b1c9@mail.gmail.com> Date: Fri, 26 Mar 2010 13:08:15 -0700 Received: by 10.220.125.83 with SMTP id x19mr969393vcr.139.1269634095099; Fri, 26 Mar 2010 13:08:15 -0700 (PDT) Message-ID: <401b577e1003261308x3a6dfe0dsf4896d34fe8cf31a@mail.gmail.com> Subject: Re: Newbie Performance Question From: Scott White To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=0016369f9dde3bb1270482b9b947 --0016369f9dde3bb1270482b9b947 Content-Type: text/plain; charset=ISO-8859-1 Yep I believe those are inserts per second. Take the last line: "811653,1666,250" I believe that's telling you that during that 10 second interval you did 1666 inserts but your overall insert rate is 811653/250 = 3246.612 inserts/sec. Timeouts may be due to your machine(s) being fully saturated? Not sure. Scott On Fri, Mar 26, 2010 at 1:00 PM, malcolm smith < malsmith@treehousesystems.com> wrote: > Ok I ran the stress test with out of box settings -- 50 threads and 1M row > inserts. It seems to get as high as 4400 ops per second and as low as 968. > Am I reading these correctly as inserts per second? > > These are results below. But is also generates timeouts and failures in > the python code like: > > Process Inserter-20: > Traceback (most recent call last): > File "/usr/lib/python2.6/multiprocessing/process.py", line 232, in > _bootstrap > self.run() > File > "/home/malsmith/dev/apache-cassandra-0.5.1-src/test/system/stress.py", line > 80, in run > self.cclient.batch_insert('Keyspace1', key, cfmap, > ConsistencyLevel.ONE) > File > "/home/malsmith/dev/apache-cassandra-0.5.1-src/interface/gen-py/cassandra/Cassandra.py", > line 583, in batch_insert > self.recv_batch_insert() > File > "/home/malsmith/dev/apache-cassandra-0.5.1-src/interface/gen-py/cassandra/Cassandra.py", > line 611, in recv_batch_insert > raise result.te > TimedOutException: TimedOutException() > self.recv_batch_insert() > File > "/home/malsmith/dev/apache-cassandra-0.5.1-src/interface/gen-py/cassandra/Cassandra.py", > line 611, in recv_batch_insert > raise result.te > TimedOutException: TimedOutException() > self.recv_batch_insert() > File > "/home/malsmith/dev/apache-cassandra-0.5.1-src/interface/gen-py/cassandra/Cassandra.py", > line 611, in recv_batch_insert > raise result.te > TimedOutException: TimedOutException() > > > total,interval_op_rate,elapsed_time > 48318,4831,10 > 58006,968,20 > 75447,1744,30 > 118266,4281,40 > 160906,4264,50 > 191501,3059,60 > 235144,4364,70 > 270721,3557,80 > 308977,3825,90 > 353383,4440,100 > 386573,3319,110 > 411550,2497,120 > 445391,3384,130 > 476990,3159,140 > 491169,1417,150 > 512848,2167,160 > 547812,3496,170 > 583997,3618,180 > 609193,2519,190 > 653878,4468,200 > 687692,3381,210 > 711378,2368,220 > 755527,4414,230 > 794985,3945,240 > 811653,1666,250 > ~ > > > ~ > On Fri, Mar 26, 2010 at 12:25 PM, Brandon Williams wrote: > >> On Fri, Mar 26, 2010 at 10:45 AM, malcolm smith < >> malsmith@treehousesystems.com> wrote: >> >>> I've been getting a feel for the performance elements of Cassandra using >>> version 0.51. I've done similar tests on HBase before, but Cassandra has >>> some very appealing aspects that I would like to pursue. >>> >>> However I'm not seeing the what seems like the common level of >>> performance others are seeing. >>> >> >> Can you test with stress.py? In 0.5, I think it was in test/stress. >> >> -Brandon >> > > --0016369f9dde3bb1270482b9b947 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Yep I believe those are inserts per second. Take the last line:

&quo= t;811653,1666,250"

I believe that's telling you that during= that 10 second interval you did 1666 inserts but your overall insert rate = is 811653/250 =3D 3246.612 inserts/sec.

Timeouts may be due to your machine(s) being fully saturated? Not sure.=

Scott

On Fri, Mar 26, 2010 at 1:0= 0 PM, malcolm smith <malsmith@treehousesystems.com> wrote:
Ok I ra= n the stress test with out of box settings -- 50 threads and 1M row inserts= . =A0 It seems to get as high as 4400 ops per second and as low as 968. =A0= Am I reading these correctly as inserts per second?

These are results below. =A0But is also generates timeouts a= nd failures in the python code like:

Process = Inserter-20:
Traceback (most recent call last):
=A0=A0F= ile "/usr/lib/python2.6/multiprocessing/process.py", line 232, in= _bootstrap
=A0=A0 =A0self.run()
=A0=A0File "/home/malsmith/dev/apa= che-cassandra-0.5.1-src/test/system/stress.py", line 80, in run
<= div>=A0=A0 =A0self.cclient.batch_insert('Keyspace1', key, cfmap, Co= nsistencyLevel.ONE)
=A0=A0File "/home/malsmith/dev/apache-cassandra-0.5.1-src/interfa= ce/gen-py/cassandra/Cassandra.py", line 583, in batch_insert
=A0=A0 =A0self.recv_batch_insert()
=A0=A0File "/home/malsmi= th/dev/apache-cassandra-0.5.1-src/interface/gen-py/cassandra/Cassandra.py&q= uot;, line 611, in recv_batch_insert
=A0=A0 =A0raise result.te
TimedOutException: TimedOutExcepti= on()
=A0=A0 =A0self.recv_batch_insert()
=A0=A0File &quo= t;/home/malsmith/dev/apache-cassandra-0.5.1-src/interface/gen-py/cassandra/= Cassandra.py", line 611, in recv_batch_insert
=A0=A0 =A0raise result.te
TimedOutException: TimedOutExcepti= on()
=A0=A0 =A0self.recv_batch_insert()
=A0=A0File &quo= t;/home/malsmith/dev/apache-cassandra-0.5.1-src/interface/gen-py/cassandra/= Cassandra.py", line 611, in recv_batch_insert
=A0=A0 =A0raise result.te
TimedOutException: TimedOutExcepti= on()


total,interval_op_r= ate,elapsed_time
48318,4831,10
58006,968,20
7= 5447,1744,30
118266,4281,40
160906,4264,50
191501,3059,60
=
235144,4364,70
270721,3557,80
308977,3825,90
=
353383,4440,100
386573,3319,110
411550,2497,120
445391,3384,130
476990,3159,140
491169,1417,150
512848,2167,160
547812,3496,170
583997,3618,180=
609193,2519,190
653878,4468,200
687692,3381,= 210
711378,2368,220
755527,4414,230
794985,3945,240
811653,1666,250
~ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0
~ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=A0
On Fri, Ma= r 26, 2010 at 12:25 PM, Brandon Williams <driftx@gmail.com> w= rote:
On Fri, Mar 26, 2010 at 10:45 AM, malcolm s= mith <malsmith@treehousesystems.com> wrote:
I've been getting a feel for the performance elements of Cassandra usin= g version 0.51. =A0I've done similar tests on HBase before, but Cassand= ra has some very appealing aspects that I would like to pursue.

However I'm not seeing the what seems like the common level of per= formance others are seeing. =A0

Can you test with stress.py? =A0In 0.5, I= think it was in test/stress.

<= div>-Brandon


--0016369f9dde3bb1270482b9b947--