Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: domain of xiaowei609@gmail.com designates
 209.85.215.44 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=ZfbU3p/26H0ig0jjd7rFbsiznulNmWuTjWWO8HHZAa57ZpHibCocab6gXpX28FpqAR
         aieMtfu+oAEaaShAjGbBHxAAe5FSnIbNqFWvM+Dn/fZtBCBuRXP9oye65cjw273EWU4x
         5DtvJMcP1cVMIGKT6S+/EQmPxyDZtnxcqXQMM=
MIME-Version: 1.0
In-Reply-To: <1B5A876D-CF35-40EB-A4C7-0491673942E2@thelastpickle.com>
References: <BANLkTi=toc9-MaWYjnX2z22pMeAMn1hgtw@mail.gmail.com>
	<1B5A876D-CF35-40EB-A4C7-0491673942E2@thelastpickle.com>
Date: Mon, 16 May 2011 11:44:50 -0400
Message-ID: <BANLkTik4R82bds_shKJVgNV-Z23CJ2j=Ng@mail.gmail.com>
Subject: Re: insert and batch_insert
From: Xiaowei Wang <xiaowei609@gmail.com>
To: user@cassandra.apache.org
Content-Type: multipart/alternative; boundary=00504502d22b33b11804a36689fc

--00504502d22b33b11804a36689fc
Content-Type: text/plain; charset=ISO-8859-1

Thanks Aaron, really help!

2011/5/16 aaron morton <aaron@thelastpickle.com>

> batch_mutate() and insert() follow the a similar execution path to a single
> insert in the server. It's not like putting multiple statements in a
> Transaction in the RDBMS.
>
> Where they do differ is that you can provide multiple columns for a row in
> a column family, and these will be applied as one operation including only
> one write to the commit log. However row you send requires a write to the
> commit log.
>
> What sort of data are you writing ? Are their multiple columns per row ?
>
> Another consideration is that each row becomes an mutation in the cluster.
> If a connection sends 1000's of rows at once all of it's mutations *could*
> momentarily fill all the available mutation workers on a node. This can slow
> down other clients connected to the cluster if they also need to write to
> that node. Watch the TPStats to see if the mutation pool has spikes in the
> pending range. You may want to reduce the batch size if clients are seeing
> high latency.
>
> Hope that helps.
>
> -----------------
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 15 May 2011, at 10:34, Xiaowei Wang wrote:
>
> > Hi,
> >
> > We use Cassandra 0.7.4 to do TPC-C data loading on ec2 nodes. The loading
> driver is written in pycassa. We test the loading speed on insert and
> batch_insert, but it seems no significant difference. I know Cassandra first
> write data to memory. But still confused why batch_insert does not quick
> than single row insert. We only batch 2000 or 3000 rows a time..
> >
> > Thanks for your help!
> >
> > Best,
> > Xiaowei
>
>

--00504502d22b33b11804a36689fc
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Thanks Aaron, really help!<br><br><div class=3D"gmail_quote">2011/5/16 aaro=
n morton <span dir=3D"ltr">&lt;<a href=3D"mailto:aaron@thelastpickle.com">a=
aron@thelastpickle.com</a>&gt;</span><br><blockquote class=3D"gmail_quote" =
style=3D"border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8=
ex; padding-left: 1ex;">
batch_mutate() and insert() follow the a similar execution path to a single=
 insert in the server. It&#39;s not like putting multiple statements in a T=
ransaction in the RDBMS.<br>
<br>
Where they do differ is that you can provide multiple columns for a row in =
a column family, and these will be applied as one operation including only =
one write to the commit log. However row you send requires a write to the c=
ommit log.<br>

<br>
What sort of data are you writing ? Are their multiple columns per row ?<br=
>
<br>
Another consideration is that each row becomes an mutation in the cluster. =
If a connection sends 1000&#39;s of rows at once all of it&#39;s mutations =
*could* momentarily fill all the available mutation workers on a node. This=
 can slow down other clients connected to the cluster if they also need to =
write to that node. Watch the TPStats to see if the mutation pool has spike=
s in the pending range. You may want to reduce the batch size if clients ar=
e seeing high latency.<br>

<br>
Hope that helps.<br>
<br>
-----------------<br>
<font color=3D"#888888">Aaron Morton<br>
Freelance Cassandra Developer<br>
@aaronmorton<br>
<a href=3D"http://www.thelastpickle.com" target=3D"_blank">http://www.thela=
stpickle.com</a><br>
</font><div><div></div><div class=3D"h5"><br>
On 15 May 2011, at 10:34, Xiaowei Wang wrote:<br>
<br>
&gt; Hi,<br>
&gt;<br>
&gt; We use Cassandra 0.7.4 to do TPC-C data loading on ec2 nodes. The load=
ing driver is written in pycassa. We test the loading speed on insert and b=
atch_insert, but it seems no significant difference. I know Cassandra first=
 write data to memory. But still confused why batch_insert does not quick t=
han single row insert. We only batch 2000 or 3000 rows a time..<br>

&gt;<br>
&gt; Thanks for your help!<br>
&gt;<br>
&gt; Best,<br>
&gt; Xiaowei<br>
<br>
</div></div></blockquote></div><br>

--00504502d22b33b11804a36689fc--