Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of rwille@fold3.com designates
 38.101.149.73 as permitted sender)
User-Agent: Microsoft-MacOutlook/14.3.6.130613
Date: Wed, 11 Dec 2013 05:40:43 -0700
Subject: Re: What is the fastest way to get data into Cassandra 2 from a Java
 application?
From: Robert Wille <rwille@fold3.com>
To: <user@cassandra.apache.org>
Message-ID: <CECDAA8A.C1CCB%rwille@fold3.com>
Thread-Topic: What is the fastest way to get data into Cassandra 2 from a Java
 application?
In-Reply-To: 
 <CA+O6_FdtkLOLMVZ81DFtxdTxON51GP20t7B_cip5MF-WH5k3Tw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: quoted-printable

I use hand-rolled batches a lot. You can get a *lot* of performance
improvement. Just make sure to sanitize your strings.

I=B9ve been wondering, what=B9s the limit, practical or hard, on the length of
a query?

Robert

On 12/11/13, 3:37 AM, "David Tinker" <david.tinker@gmail.com> wrote:

>Yes thats what I found.
>
>This is faster:
>
>for (int i =3D 0; i < 1000; i++) session.execute("INSERT INTO
>test.wibble (id, info) VALUES ('${"" + i}', '${"aa" + i}')")
>
>Than this:
>
>def ps =3D session.prepare("INSERT INTO test.wibble (id, info) VALUES (?,
>?)")
>for (int i =3D 0; i < 1000; i++) session.execute(ps.bind(["" + i, "aa" +
>i] as Object[]))
>
>This is the fastest option of all (hand rolled batch):
>
>StringBuilder b =3D new StringBuilder()
>b.append("BEGIN UNLOGGED BATCH\n")
>for (int i =3D 0; i < 1000; i++) {
>    b.append("INSERT INTO ").append(ks).append(".wibble (id, info)
>VALUES ('").append(i).append("','")
>            .append("aa").append(i).append("')\n")
>}
>b.append("APPLY BATCH\n")
>session.execute(b.toString())
>
>
>On Wed, Dec 11, 2013 at 10:56 AM, Sylvain Lebresne <sylvain@datastax.com>
>wrote:
>>
>>> This loop takes 2500ms or so on my test cluster:
>>>
>>> PreparedStatement ps =3D session.prepare("INSERT INTO perf_test.wibble
>>> (id, info) VALUES (?, ?)")
>>> for (int i =3D 0; i < 1000; i++) session.execute(ps.bind("" + i, "aa" +
>>>i));
>>>
>>> The same loop with the parameters inline is about 1300ms. It gets
>>> worse if there are many parameters.
>>
>>
>> Do you mean that:
>>   for (int i =3D 0; i < 1000; i++)
>>       session.execute("INSERT INTO perf_test.wibble (id, info) VALUES
>>(" + i
>> + ", aa" + i + ")");
>> is twice as fast as using a prepared statement? And that the difference
>> is even greater if you add more columns than "id" and "info"?
>>
>> That would certainly be unexpected, are you sure you're not
>>re-preparing the
>> statement every time in the loop?
>>
>> --
>> Sylvain
>>
>>> I know I can use batching to
>>> insert all the rows at once but thats not the purpose of this test. I
>>> also tried using session.execute(cql, params) and it is faster but
>>> still doesn't match inline values.
>>>
>>> Composing CQL strings is certainly convenient and simple but is there
>>> a much faster way?
>>>
>>> Thanks
>>> David
>>>
>>> I have also posted this on Stackoverflow if anyone wants the points:
>>>
>>>=20
>>>http://stackoverflow.com/questions/20491090/what-is-the-fastest-way-to-g
>>>et-data-into-cassandra-2-from-a-java-application
>>
>>
>
>
>
>--=20
>http://qdb.io/ Persistent Message Queues With Replay and #RabbitMQ
>Integration