Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of milindparikh@gmail.com
 designates 209.85.215.44 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=kgdYtasPaYOMddf1zeeEMcbf7a9P3MoLrIPdGiHDVJt5wa0cFJg3+oWUN2tF4h9M4P
         fMrMTjPWpT+bcs42tM+bs+27KjOi2tQqSZO6wY8iYEwgrAOcJPeCZALvXkX3BH3/U5yI
         PtIm+BrdyqVr4dzCrpe3EXPbrh8sCUV7Ow3Bo=
MIME-Version: 1.0
In-Reply-To: <BANLkTim9SHCc57fNQ6qCp+FNAsCaxbUMXA@mail.gmail.com>
References: <BANLkTina+ivJ+dc87whKHo9m5DDBNDQP-Q@mail.gmail.com>
	<8BEFDE0B-9EA3-44B2-A9B7-E077F138AEBD@thelastpickle.com>
	<BANLkTim9SHCc57fNQ6qCp+FNAsCaxbUMXA@mail.gmail.com>
Date: Sun, 22 May 2011 15:53:21 -0400
Message-ID: <BANLkTimkg_GTXC5CpfwcBE5GHJ++u7NJbQ@mail.gmail.com>
Subject: Re: rainbird question (why is the 1minute buffer needed?)
From: Milind Parikh <milindparikh@gmail.com>
To: user@cassandra.apache.org
Content-Type: multipart/alternative; boundary=90e6ba5bbaf706ec5a04a3e2b5e2

--90e6ba5bbaf706ec5a04a3e2b5e2
Content-Type: text/plain; charset=ISO-8859-1

I believe that the key reason is souped up performance for most recent data.
And yes, "an intelligent flush" leaves you vulnerable to some data loss.

/***********************
sent from my android...please pardon occasional typos as I respond @ the
speed of thought
************************/

On May 22, 2011 11:01 AM, "Yang" <teddyyyy123@gmail.com> wrote:

Thanks,

I did read through that pdf doc, and went through the counters code in
0.8-rc2, I think I understand the logic in that code.

in my hypothetical implementation, I am not suggesting to overstep the
complicated logic in counters code, since the extra module will still
need to enter the increment through StorageProxy.mutate(
My_counter.delta=1 ) , so that the logical clock is still handled by
the Counters code.

 the only difference is, as you said,
that rainbird collapses many +1 deltas. but my claim is that in fact
this "collapsing" is already done by cassandra since the write always
hit the memtable  first,
so collapsing in Cassandra memtable vs collapsing in rainbird  memory
takes the same time, while rainbird introduces an extra level of
caching (I am strongly suspecting that rainbird is vulnerable to
losing up to 1minute's worth of data , if the rainbird dies before the
writes are flushed to cassandra ---- unless it does implement its own
commit log, but that is kind of  re-implementing many of the wheels in
Cassandra ....)


I thought at one time probably the reason was because that from one
given url, rainbird needs to create writes on many keys, so that they
keys need to go to different
Cassandra nodes. but later I found that this can also be done in a
module on the coordinator, since the client request first hits a
coordinator, instead of the data node, in fact, in a multi-insert
case, the coordinator already sends the request to multiple data
nodes. the extra module I am proposing simply translates a single
insert into multi-insert, and then cassandra takes over from there


Thanks
Yang


On Sun, May 22, 2011 at 3:47 AM, aaron morton <aaron@thelastpickle.com>
wrote:
>  The implementatio...

--90e6ba5bbaf706ec5a04a3e2b5e2
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<p>I believe that the key reason is souped up performance for most recent d=
ata.<br>
And yes, &quot;an intelligent flush&quot; leaves you vulnerable to some dat=
a loss.</p>
<p>/***********************<br>
sent from my android...please pardon occasional typos as I respond @ the sp=
eed of thought<br>
************************/</p>
<p><blockquote type=3D"cite">On May 22, 2011 11:01 AM, &quot;Yang&quot; &lt=
;<a href=3D"mailto:teddyyyy123@gmail.com">teddyyyy123@gmail.com</a>&gt; wro=
te:<br><br>Thanks,<br>
<br>
I did read through that pdf doc, and went through the counters code in<br>
0.8-rc2, I think I understand the logic in that code.<br>
<br>
in my hypothetical implementation, I am not suggesting to overstep the<br>
complicated logic in counters code, since the extra module will still<br>
need to enter the increment through StorageProxy.mutate(<br>
My_counter.delta=3D1 ) , so that the logical clock is still handled by<br>
the Counters code.<br>
<br>
=A0the only difference is, as you said,<br>
that rainbird collapses many +1 deltas. but my claim is that in fact<br>
this &quot;collapsing&quot; is already done by cassandra since the write al=
ways<br>
hit the memtable =A0first,<br>
so collapsing in Cassandra memtable vs collapsing in rainbird =A0memory<br>
takes the same time, while rainbird introduces an extra level of<br>
caching (I am strongly suspecting that rainbird is vulnerable to<br>
losing up to 1minute&#39;s worth of data , if the rainbird dies before the<=
br>
writes are flushed to cassandra ---- unless it does implement its own<br>
commit log, but that is kind of =A0re-implementing many of the wheels in<br=
>
Cassandra ....)<br>
<br>
<br>
I thought at one time probably the reason was because that from one<br>
given url, rainbird needs to create writes on many keys, so that they<br>
keys need to go to different<br>
Cassandra nodes. but later I found that this can also be done in a<br>
module on the coordinator, since the client request first hits a<br>
coordinator, instead of the data node, in fact, in a multi-insert<br>
case, the coordinator already sends the request to multiple data<br>
nodes. the extra module I am proposing simply translates a single<br>
insert into multi-insert, and then cassandra takes over from there<br>
<br>
<br>
Thanks<br>
<font color=3D"#888888">Yang<br>
</font><p><font color=3D"#500050"><br>On Sun, May 22, 2011 at 3:47 AM, aaro=
n morton &lt;<a href=3D"mailto:aaron@thelastpickle.com">aaron@thelastpickle=
.com</a>&gt; wrote:<br>&gt; =A0The implementatio...</font></p></blockquote>=
</p>

--90e6ba5bbaf706ec5a04a3e2b5e2--