Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: domain of shahab.yunus@gmail.com
 designates 209.85.214.49 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <7743523DB3AB4E1BB444AEBC78D9353A@CompudictedHP>
References: 
 <CAJV_UYeOdZfKOzCCyesZXODCDmStkdGgafjUBvWqpeeUJ2HYvA@mail.gmail.com>
	<7743523DB3AB4E1BB444AEBC78D9353A@CompudictedHP>
Date: Wed, 19 Jun 2013 08:40:34 -0400
Message-ID: 
 <CAEo-6+Ss6XjCsj9EUcYUPtZnuwg10TvL3Dn1de1cNh2ugxEL5Q@mail.gmail.com>
Subject: Re: Dropped mutation messages
From: Shahab Yunus <shahab.yunus@gmail.com>
To: user@cassandra.apache.org
Content-Type: multipart/alternative; boundary=14dae9c0981ec9b8ee04df81225f

--14dae9c0981ec9b8ee04df81225f
Content-Type: text/plain; charset=ISO-8859-1

Hello Arthur,

What do you mean by "The queries need to be lightened"?

Thanks,
Shahb


On Tue, Jun 18, 2013 at 8:47 PM, Arthur Zubarev <Arthur.Zubarev@aol.com>wrote:

>   Cem hi,
>
> as per http://wiki.apache.org/cassandra/FAQ#dropped_messages
>
>
> Internode messages which are received by a node, but do not get not to be
> processed within rpc_timeout are dropped rather than processed. As the
> coordinator node will no longer be waiting for a response. If the
> Coordinator node does not receive Consistency Level responses before the
> rpc_timeout it will return a TimedOutException to the client. If the
> coordinator receives Consistency Level responses it will return success to
> the client.
>
> For MUTATION messages this means that the mutation was not applied to all
> replicas it was sent to. The inconsistency will be repaired by Read Repair
> or Anti Entropy Repair.
>
> For READ messages this means a read request may not have completed.
>
> Load shedding is part of the Cassandra architecture, if this is a
> persistent issue it is generally a sign of an overloaded node or cluster.
>
> By the way, I am on C* 1.2.4 too in dev mode, after having my node filled
> with 400 GB I started getting RPC timeouts on large data retrievals, so in
> short, you may need to revise how you query.
>
> The queries need to be lightened
>
> /Arthur
>
>  *From:* cem <cayiroglu@gmail.com>
> *Sent:* Tuesday, June 18, 2013 1:12 PM
> *To:* user@cassandra.apache.org
> *Subject:* Dropped mutation messages
>
>  Hi All,
>
> I have a cluster of 5 nodes with C* 1.2.4.
>
> Each node has 4 disks 1 TB each.
>
> I see  a lot of dropped messages after it stores 400 GB  per disk. (1.6 TB
> per node).
>
> The recommendation was 500 GB max per node before 1.2.  Datastax says that
> we can store terabytes of data per node with 1.2.
> http://www.datastax.com/docs/1.2/cluster_architecture/cluster_planning
>
> Do I need to enable anything to leverage from 1.2? Do you have any other
> advice?
>
> What should be the path to investigate this?
>
> Thanks in advance!
>
> Best Regards,
> Cem.
>
>
>

--14dae9c0981ec9b8ee04df81225f
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hello Arthur,<div><br></div><div style>What do you mean by=
 &quot;<span style=3D"font-family:Calibri;font-size:16px">The queries need =
to be lightened&quot;?</span></div><div style><span style=3D"font-family:Ca=
libri;font-size:16px"><br>
</span></div><div style><span style=3D"font-family:Calibri;font-size:16px">=
Thanks,</span></div><div style><span style=3D"font-family:Calibri;font-size=
:16px">Shahb</span></div></div><div class=3D"gmail_extra"><br><br><div clas=
s=3D"gmail_quote">
On Tue, Jun 18, 2013 at 8:47 PM, Arthur Zubarev <span dir=3D"ltr">&lt;<a hr=
ef=3D"mailto:Arthur.Zubarev@aol.com" target=3D"_blank">Arthur.Zubarev@aol.c=
om</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"marg=
in:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">

<div dir=3D"ltr">
<div dir=3D"ltr">
<div style=3D"font-size:12pt;font-family:&#39;Calibri&#39;">
<div>Cem hi,</div>
<div>=A0</div>
<div>as per <a href=3D"http://wiki.apache.org/cassandra/FAQ#dropped_message=
s" target=3D"_blank"><font face=3D"Times New Roman">http://wiki.apache.org/=
cassandra/FAQ#dropped_messages</font></a></div>
<div>=A0</div>
<p><font face=3D"Arial">Internode messages which are received by a node, bu=
t do not get not=20
to be processed within rpc_timeout are dropped rather than processed. As th=
e=20
coordinator node will no longer be waiting for a response. If the Coordinat=
or=20
node does not receive Consistency Level responses before the rpc_timeout it=
 will=20
return a TimedOutException to the client. If the coordinator receives=20
Consistency Level responses it will return success to the client.<span></sp=
an></font><span></span></p>
<p><font face=3D"Arial">For MUTATION messages this means that the mutation =
was not applied to=20
all replicas it was sent to. The inconsistency will be repaired by Read Rep=
air=20
or Anti Entropy Repair.<span></span></font><span></span></p>
<p><font face=3D"Arial">For READ messages this means a read request may not=
 have=20
completed.<span></span></font><span></span></p>
<p><font face=3D"Arial">Load shedding is part of the Cassandra architecture=
, if this is a=20
persistent issue it is generally a sign of an overloaded node or=20
cluster.</font></p>
<p>By=20
the way, I am on C* 1.2.4 too in dev mode, after having my node filled with=
 400=20
GB I started getting RPC timeouts on large data retrievals, so in short, yo=
u may=20
need to revise how you query.</p>
<p>The=20
queries need to be lightened </p>
<p>/Arthur</p>
<div style=3D"font-style:normal;font-size:small;display:inline;text-decorat=
ion:none;font-family:&#39;Calibri&#39;;font-weight:normal">
<div style=3D"FONT:10pt tahoma">
<div>=A0</div>
<div style=3D"BACKGROUND:#f5f5f5">
<div><b>From:</b> <a title=3D"cayiroglu@gmail.com" href=3D"mailto:cayiroglu=
@gmail.com" target=3D"_blank">cem</a> </div>
<div><b>Sent:</b> Tuesday, June 18, 2013 1:12 PM</div>
<div><b>To:</b> <a title=3D"user@cassandra.apache.org" href=3D"mailto:user@=
cassandra.apache.org" target=3D"_blank">user@cassandra.apache.org</a> </div=
>
<div><b>Subject:</b> Dropped mutation messages</div></div></div>
<div>=A0</div></div>
<div style=3D"font-style:normal;font-size:small;display:inline;text-decorat=
ion:none;font-family:&#39;Calibri&#39;;font-weight:normal">
<div dir=3D"ltr">Hi All,=20
<div>=A0</div>
<div>I have a cluster of 5 nodes with C* 1.2.4.</div>
<div>=A0</div>
<div>Each node has 4 disks 1 TB each.</div>
<div>=A0</div>
<div>I see=A0 a lot of dropped messages after it stores 400 GB=A0 per=20
disk. (1.6 TB per node).</div>
<div>=A0</div>
<div>The recommendation was 500 GB max per node before 1.2.=A0 Datastax say=
s=20
that we can store terabytes of data per node with 1.2.</div>
<div><a href=3D"http://www.datastax.com/docs/1.2/cluster_architecture/clust=
er_planning" target=3D"_blank">http://www.datastax.com/docs/1.2/cluster_arc=
hitecture/cluster_planning</a></div>
<div>=A0</div>
<div>Do I need to enable anything to leverage from 1.2? Do you have any oth=
er=20
advice?<br></div>
<div>=A0</div>
<div>What should be the path to investigate this?</div>
<div>=A0</div>
<div>Thanks in advance! </div>
<div>=A0</div>
<div>Best Regards,</div>
<div>Cem.</div>
<div>=A0</div>
<div>=A0</div></div></div></div></div></div>
</blockquote></div><br></div>

--14dae9c0981ec9b8ee04df81225f--