Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; c=nofws; d=thelastpickle.com; h=to:cc:from
	:subject:date:message-id:content-type:mime-version:in-reply-to;
	 q=dns; s=thelastpickle.com; b=UQJxnT8SLx2KXMC+TDFh3c4zavkzhag5n
	TzLCkANcxHzssO/aO8RnyVEOrrZ6F2yTLItyfX54iOH1NHeglFlaZy6gi6qGy7Np
	Q7JwP0R4nqBc76i+Q9SQyD46JARuvLNGo1IdXjCe7QVVq88lwvFt6B0EWC9KFges
	PLO6Oh3W50=
To: user@cassandra.apache.org
Cc: user@cassandra.apache.org
From: Aaron Morton <aaron@thelastpickle.com>
Subject: Re: Cassandra operation success ratio survey results
Date: Tue, 21 Sep 2010 21:09:38 GMT
Message-id: <2780cb82-a638-ee2c-839d-d295b5406a6e@me.com>
Content-Type: multipart/alternative;
 boundary=Apple-Webmail-42--d05cec93-f7e8-4ebc-9709-b8734e79bbb6
MIME-Version: 1.0
In-Reply-To: <AANLkTimbaW08u6AE=w54ZUY2Hob+q8TnYg8j2CqLreAO@mail.gmail.com>


--Apple-Webmail-42--d05cec93-f7e8-4ebc-9709-b8734e79bbb6
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
    charset=ISO-8859-1;
    format=flowed

Thanks for this, really interesting stuff.=A0=0A=0AJust to make sure I'm=A0=
understanding=A0it, this is for PHP clients witha =A01 second timeout and =
retry is to a different=A0=0Anode in the cluster with the same timeout.=A0=
=0A=0AHave you enabled the Dynamic Snitch ?=A0http://www.riptano.com/blog/=
whats-new-cassandra-065=0A=0AAaron=0A=0AOn 22 Sep, 2010,at 01:29 AM, Juho =
M=E4kinen <juho.makinen@gmail.com> wrote:=0A=0AIt's known that compaction =
hurts the node performance so that it might=0Amiss some requests. That's w=
hy it's important to handle these=0Asituations and the client needs to ret=
ry the operation into another=0Aworking host. We have been storing perform=
ance data from each=0Acassandra request which we do into our five node cas=
sandra production=0Acluster.=0A=0AWe log the retry count and request type =
into our data warehouse=0Asolution and I've now extracted the data from a =
10 day period and=0Acalculated how many retry requests is needed so that t=
he results can=0Abe obtained. The following chart tells how many time an o=
peration had=0Ato be retried until it was successfully completed. The perc=
ents tells=0Athe probability like that "the request will be successful wit=
h the=0Afirst try in 99.933 % times."=0A=0ATotal amount of operations: 94 =
682 251 within 10 days.=0A=0ARetry times | operations | percentage from to=
tal operations=0A0 | 94618468 | 99.93263 %=0A1 | 56688 | 0.05987 %=0A2 | 5=
018 | 0.00529 %=0A3 | 1359 | 0.00144 %=0A4 | 111 | 0.00012 %=0A5 | 25 | 0.=
00003 %=0A=0AThere were also few operations which needed more than five re=
tries, so=0Apreparing to try up to ten times is not a bad idea.=0A=0AThe c=
luster users 0.6.5 with RF=3D3. Each operation is executed until it=0Asucc=
eeds or until 10 retries using this php wrapper=0Ahttp://github.com/dynamo=
id/cassandra-utilities=0A=0AHave others found similar results? Please disc=
uss :)=0A=0A- Juho M=E4kinen=0A
--Apple-Webmail-42--d05cec93-f7e8-4ebc-9709-b8734e79bbb6
Content-Type: multipart/related;
    type="text/html";
    boundary=Apple-Webmail-86--d05cec93-f7e8-4ebc-9709-b8734e79bbb6


--Apple-Webmail-86--d05cec93-f7e8-4ebc-9709-b8734e79bbb6
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
    charset=ISO-8859-1;

<html><body><div>Thanks for this, really interesting stuff.&nbsp;</div><div><br></div>=
<div>Just to make sure I'm&nbsp;understanding&nbsp;it, this is for PHP cli=
ents witha &nbsp;1 second timeout and retry is to a different&nbsp;</div><=
div>node in the cluster with the same timeout.&nbsp;</div><div><br></div><=
div>Have you enabled the Dynamic Snitch ?&nbsp;<a href=3D"http://www.ripta=
no.com/blog/whats-new-cassandra-065">http://www.riptano.com/blog/whats-new=
-cassandra-065</a></div><div><meta http-equiv=3D"content-type" content=3D"=
text/html; charset=3Dutf-8"></div><div><br></div><div>Aaron</div><div><br>=
On 22 Sep, 2010,at 01:29 AM, Juho M=E4kinen &lt;juho.makinen@gmail.com&gt;=
 wrote:<br><br></div><div><blockquote type=3D"cite"><div><div class=3D"_st=
retch">It's known that compaction hurts the node performance so that it mi=
ght<br>=0Amiss some requests. That's why it's important to handle these<br=
>=0Asituations and the client needs to retry the operation into another<br=
>=0Aworking host. We have been storing performance data from each<br>=0Aca=
ssandra request which we do into our five node cassandra production<br>=0A=
cluster.<br>=0A<br>=0AWe log the retry count and request type into our dat=
a warehouse<br>=0Asolution and I've now extracted the data from a 10 day p=
eriod and<br>=0Acalculated how many retry requests is needed so that the r=
esults can<br>=0Abe obtained. The following chart tells how many time an o=
peration had<br>=0Ato be retried until it was successfully completed. The =
percents tells<br>=0Athe probability like that "the request will be succes=
sful with the<br>=0Afirst try in 99.933 % times."<br>=0A<br>=0ATotal amoun=
t of operations: 94 682 251 within 10 days.<br>=0A<br>=0ARetry times | ope=
rations | percentage from total operations<br>=0A          0 |  94618468  =
| 99.93263 %<br>=0A          1 |     56688  |  0.05987 %<br>=0A          2=
 |      5018  |  0.00529 %<br>=0A          3 |      1359  |  0.00144 %<br>=
=0A          4 |       111  |  0.00012 %<br>=0A          5 |       25   | =
 0.00003 %<br>=0A<br>=0AThere were also few operations which needed more t=
han five retries, so<br>=0Apreparing to try up to ten times is not a bad i=
dea.<br>=0A<br>=0AThe cluster users 0.6.5 with RF=3D3. Each operation is e=
xecuted until it<br>=0Asucceeds or until 10 retries using this php wrapper=
<br>=0A<a href=3D"http://github.com/dynamoid/cassandra-utilities" _mce_hre=
f=3D"http://github.com/dynamoid/cassandra-utilities">http://github.com/dyn=
amoid/cassandra-utilities</a><br>=0A<br>=0AHave others found similar resul=
ts? Please discuss :)<br>=0A<br>=0A - Juho M=E4kinen<br>=0A</div></div></b=
lockquote></div></body></html>
--Apple-Webmail-86--d05cec93-f7e8-4ebc-9709-b8734e79bbb6--

--Apple-Webmail-42--d05cec93-f7e8-4ebc-9709-b8734e79bbb6--