incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From graham sanderson <gra...@vast.com>
Subject Re: What is the fastest way to get data into Cassandra 2 from a Java application?
Date Tue, 10 Dec 2013 11:17:51 GMT
I should probably give you a number which is about 300 meg / s via thrift api and use 1mb batches

On Dec 10, 2013, at 5:14 AM, graham sanderson <graham@vast.com> wrote:

> Perhaps not the way forward, however I can bulk insert data via astyanax at a rate that
maxes out our (fast) networks. That said for our next release (of this part of our product
- our other current is node.js via binary protocol) we will be looking at insert speed via
java driver, and also alternative scala/java implementations of the binary protocol.
> 
> On Dec 10, 2013, at 4:49 AM, David Tinker <david.tinker@gmail.com> wrote:
> 
>> I have tried the DataStax Java driver and it seems the fastest way to
>> insert data is to compose a CQL string with all parameters inline.
>> 
>> This loop takes 2500ms or so on my test cluster:
>> 
>> PreparedStatement ps = session.prepare("INSERT INTO perf_test.wibble
>> (id, info) VALUES (?, ?)")
>> for (int i = 0; i < 1000; i++) session.execute(ps.bind("" + i, "aa" + i));
>> 
>> The same loop with the parameters inline is about 1300ms. It gets
>> worse if there are many parameters. I know I can use batching to
>> insert all the rows at once but thats not the purpose of this test. I
>> also tried using session.execute(cql, params) and it is faster but
>> still doesn't match inline values.
>> 
>> Composing CQL strings is certainly convenient and simple but is there
>> a much faster way?
>> 
>> Thanks
>> David
>> 
>> I have also posted this on Stackoverflow if anyone wants the points:
>> http://stackoverflow.com/questions/20491090/what-is-the-fastest-way-to-get-data-into-cassandra-2-from-a-java-application
> 


Mime
View raw message