Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: domain of driftx@gmail.com designates
 209.85.214.44 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type;
        b=QO+tQ2g6H6AXoUVxdKNlu8/rMwKtMVc5RPmZZApZBS6NJYxeIb7YXcy2w0SxONGnKZ
         1Kyh1qG2EeT0emcD5WbFgKGL7ov1qqn4Rgx4lls620cjtoGjCazdJuiiF1uQ6ij400rC
         glRPO/cW0wcKpNKUoXB4xRHn0SE+Dl6Cmw1aY=
MIME-Version: 1.0
In-Reply-To: <AANLkTik4LcU28SRmy9tkgEopUCzs++sQ_TaD7jmyNnk=@mail.gmail.com>
References: <AANLkTik4LcU28SRmy9tkgEopUCzs++sQ_TaD7jmyNnk=@mail.gmail.com>
From: Brandon Williams <driftx@gmail.com>
Date: Thu, 3 Feb 2011 19:22:51 -0600
Message-ID: <AANLkTi=aUi=RrAjHSC-dTuKUndA4YvUAOeFf_T0tfZO6@mail.gmail.com>
Subject: Re: Problems with Python Stress Test
To: user@cassandra.apache.org
Content-Type: multipart/alternative; boundary=001485f78f6ebdb621049b6ab90d

--001485f78f6ebdb621049b6ab90d
Content-Type: text/plain; charset=ISO-8859-1

On Thu, Feb 3, 2011 at 7:02 PM, Sameer Farooqui <cassandralabs@gmail.com>wrote:

> Hi guys,
>
> I was playing around with the stress.py test this week and noticed a few
> things.
>
> 1) Progress-interval does not always work correctly. I set it to 5 in the
> example below, but am instead getting varying intervals:
>

Generally indicates that the client machine is being overloaded in my
experience.

2) The key_rate and op_rate doesn't seem to be calculated correctly. Also,
> what is the difference between the interval_key_rate and the
> interval_op_rate? For example in the example above, the first row shows 6662
> keys inserted in 5 seconds and 6662 / 5 = 1332, which matches the
> interval_op_rate.
>

There should be no difference unless you're doing range slices, but IPC
timing makes them vary somewhat.

3) If I write x KB to Cassandra with py_stress, the used disk space doesn't
> grow by x after the test. In the example below I tried to write 500,000 keys
> * 32 bytes * 5 columns = 78,125 kilobytes of data to the database. When I
> checked the amount of disk space used after the test it actually grew by
> 2,684,920 - 2,515,864 = 169,056 kilobytes. Is this because perhaps the
> commit log got duplicate copies of the data as the SSTables?
>

Commitlogs could be part of it, you're not factoring in the column names,
and then there's index and bloom filter overhead.

Use contrib/stress on 0.7 instead.

-Brandon

--001485f78f6ebdb621049b6ab90d
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div class=3D"gmail_quote">On Thu, Feb 3, 2011 at 7:02 PM, Sameer Farooqui =
<span dir=3D"ltr">&lt;<a href=3D"mailto:cassandralabs@gmail.com">cassandral=
abs@gmail.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" st=
yle=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">

Hi guys,<br><br>I was playing around with the stress.py test this week and =
noticed a few things.<br><br>1) Progress-interval does not always work corr=
ectly. I set it to 5 in the example below, but am instead getting varying i=
ntervals:<br>

</blockquote><div><br></div><div>Generally indicates that the client machin=
e is being overloaded in my experience.=A0</div><div><br></div><blockquote =
class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid=
;padding-left:1ex;">

2) The key_rate and op_rate doesn&#39;t seem to be calculated correctly. Al=
so, what is the difference between the interval_key_rate and the interval_o=
p_rate? For example in the example above, the first row shows 6662 keys ins=
erted in 5 seconds and 6662 / 5 =3D 1332, which matches the=A0 interval_op_=
rate.<br>

</blockquote><div><br></div><div>There should be no difference unless you&#=
39;re doing range slices, but IPC timing makes them vary somewhat.=A0</div>=
<div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex=
;border-left:1px #ccc solid;padding-left:1ex;">

3) If I write x KB to Cassandra with py_stress, the used disk space doesn&#=
39;t grow by x after the test. In the example below I tried to write 500,00=
0 keys * 32 bytes * 5 columns =3D 78,125 kilobytes of data to the database.=
 When I checked the amount of disk space used after the test it actually gr=
ew by 2,684,920 - 2,515,864 =3D 169,056 kilobytes. Is this because perhaps =
the commit log got duplicate copies of the data as the SSTables?<br>

</blockquote><div><br></div><div>Commitlogs could be part of it, you&#39;re=
 not factoring in the column names, and then there&#39;s index and bloom fi=
lter overhead.</div><div>=A0</div><div>Use contrib/stress on 0.7 instead.</=
div>

<div><br></div><div>-Brandon</div></div>

--001485f78f6ebdb621049b6ab90d--