Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
MIME-Version: 1.0
In-Reply-To: <5576AA26.3070007@ericsson.com>
References: 
 <CA+ShD0dxSpXsd=mE-22cvhwS08yYwMKAwoy0ARTyTE4Z55zP1A@mail.gmail.com>
	<5576AA26.3070007@ericsson.com>
Date: Tue, 9 Jun 2015 02:19:34 -0700
Message-ID: 
 <CA+ShD0fWCbHbeuYb1_TFtaL6W+KLWJTsr3zV1nf0VQOkFFr66w@mail.gmail.com>
Subject: Re: Cassandra Insert Rate
From: SHANKAR REDDY <sankara.telukutla@gmail.com>
To: user@cassandra.apache.org
Content-Type: multipart/alternative; boundary=90e6ba6e8642bcfb7b0518124152

--90e6ba6e8642bcfb7b0518124152
Content-Type: text/plain; charset=UTF-8

Thanks for BR for the quick response on this and Appreciate it.

That helps for Batch load.

If 10 million users are inserting new records at a time ( 1 record for one
user ) then how  do we increase the same. My sample program assumes request
from 10 million records.

-Shankar


On Tue, Jun 9, 2015 at 1:56 AM, Marcus Olsson <marcus.olsson@ericsson.com>
wrote:

>  Hi Shankar,
>
> I would say:
> * Prepared statements to avoid sending the whole statement with every
> query and instead just send the values.
> * Using session.executeAsync() to improve concurrency.
>
> So you would start by creating a prepared statement, something like:
>
> PreparedStatement ps = session.prepare("INSERT INTO ks.tb
> (key1,data1,data2) VALUES (?,?,?)"); // Only done once
>
> And then in loadData():
> session.executeAsync(ps.bind("key", "1", "2"));
>
> The executeAsync() does not wait for a response for the query, so that
> should probably be done elsewhere(see the link below for how you can get
> the results back).
>
> http://www.datastax.com/dev/blog/java-driver-async-queries
>
> BR
> Marcus Olsson
>
>
> On 06/09/2015 10:27 AM, SHANKAR REDDY wrote:
>
> Team,
> I have a sample insert query which loads around 10 million records and
> found that the insert rate is around 1500 per second.  This is very slow.
>
>  The Source code I am using available at the below location. I am using
> the very latest version 2.1.6 with default seetings  and single node VM
> machine with 20GM RAM and 100 GM SSD disk.
>
>
> https://github.com/shankar-reddy/CassandraSandbox/blob/master/src/main/java/com/itreddys/cassandra/example/BulkLoadTest.java
>
>  Please suggest on insert rate improvement.
>
>  -Shankar
>
>
>

--90e6ba6e8642bcfb7b0518124152
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Thanks for BR for the quick response on this and Appr=
eciate it.</div><div><br></div><div>That helps for Batch load.</div><div><b=
r></div><div>If 10 million users are inserting new records at a time ( 1 re=
cord for one user ) then how =C2=A0do we increase the same. My sample progr=
am assumes request from 10 million records.<br></div><div class=3D"gmail_ex=
tra"><br clear=3D"all"><div><div class=3D"gmail_signature"><div dir=3D"ltr"=
><div><div dir=3D"ltr"><div dir=3D"ltr">-Shankar<br><div><br></div></div></=
div></div></div></div></div>
<br><div class=3D"gmail_quote">On Tue, Jun 9, 2015 at 1:56 AM, Marcus Olsso=
n <span dir=3D"ltr">&lt;<a href=3D"mailto:marcus.olsson@ericsson.com" targe=
t=3D"_blank">marcus.olsson@ericsson.com</a>&gt;</span> wrote:<br><blockquot=
e class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc sol=
id;padding-left:1ex">
 =20
   =20
 =20
  <div text=3D"#000000" bgcolor=3D"#FFFFFF">
    Hi Shankar,<br>
    <br>
    I would say:<br>
    * Prepared statements to avoid sending the whole statement with
    every query and instead just send the values.<br>
    * Using session.executeAsync() to improve concurrency.<br>
    <br>
    So you would start by creating a prepared statement, something like:<br=
>
    <br>
    PreparedStatement ps =3D session.prepare(&quot;INSERT INTO ks.tb
    (key1,data1,data2) VALUES (?,?,?)&quot;); // Only done once<br>
    <br>
    And then in loadData():<br>
    session.executeAsync(ps.bind(&quot;key&quot;, &quot;1&quot;, &quot;2&qu=
ot;));<br>
    <br>
    The executeAsync() does not wait for a response for the query, so
    that should probably be done elsewhere(see the link below for how
    you can get the results back).<br>
    <br>
    <a href=3D"http://www.datastax.com/dev/blog/java-driver-async-queries" =
target=3D"_blank">http://www.datastax.com/dev/blog/java-driver-async-querie=
s</a><br>
    <br>
    BR<span class=3D"HOEnZb"><font color=3D"#888888"><br>
    Marcus Olsson</font></span><div><div class=3D"h5"><br>
    <br>
    <div>On 06/09/2015 10:27 AM, SHANKAR REDDY
      wrote:<br>
    </div>
    <blockquote type=3D"cite">
     =20
      <div dir=3D"ltr">Team,
        <div>I have a sample insert query which loads around 10 million
          records and found that the insert rate is around 1500 per
          second.=C2=A0 This is very slow.</div>
        <div><br>
        </div>
        <div>The Source code I am using available at the below location.
          I am using the very latest version 2.1.6 with default seetings
          =C2=A0and single node VM machine with 20GM RAM and 100 GM SSD
          disk.=C2=A0</div>
        <div><br>
        </div>
        <div><a href=3D"https://github.com/shankar-reddy/CassandraSandbox/b=
lob/master/src/main/java/com/itreddys/cassandra/example/BulkLoadTest.java" =
target=3D"_blank">https://github.com/shankar-reddy/CassandraSandbox/blob/ma=
ster/src/main/java/com/itreddys/cassandra/example/BulkLoadTest.java</a></di=
v>
        <div>
          <div>
            <div>
              <div dir=3D"ltr">
                <div dir=3D"ltr">
                  <div dir=3D"ltr"><br>
                  </div>
                  <div>Please suggest on insert rate improvement.=C2=A0</di=
v>
                  <div dir=3D"ltr"><br>
                  </div>
                  <div dir=3D"ltr">-Shankar<br>
                    <div><br>
                    </div>
                  </div>
                </div>
              </div>
            </div>
          </div>
        </div>
      </div>
    </blockquote>
    <br>
  </div></div></div>

</blockquote></div><br></div></div>

--90e6ba6e8642bcfb7b0518124152--