ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Ozerov <voze...@gridgain.com>
Subject Re: Batch DML queries design discussion
Date Thu, 08 Dec 2016 11:16:18 GMT
Hi Alex,

To my understanding there are two possible approaches to batching in JDBC
layer:

1) Rely on default batching API. Specifically
*PreparedStatement.addBatch()* [1]
and others. This is nice and clear API, users are used to it, and it's
adoption will minimize user code changes when migrating from other JDBC
sources. We simply copy updates locally and then execute them all at once
with only a single network hop to servers. *IgniteDataStreamer* can be used
underneath.

2) Or we can have separate connection flag which will move all
INSERT/UPDATE/DELETE statements through streamer.

I prefer the first approach

Also we need to keep in mind that data streamer has poor performance when
adding single key-value pairs due to high overhead on concurrency and other
bookkeeping. Instead, it is better to pre-batch key-value pairs before
giving them to streamer.

Vladimir.

[1]
https://docs.oracle.com/javase/8/docs/api/java/sql/PreparedStatement.html#addBatch--

On Thu, Dec 8, 2016 at 1:21 PM, Alexander Paschenko <
alexander.a.paschenko@gmail.com> wrote:

> Hello Igniters,
>
> One of the major improvements to DML has to be support of batch
> statements. I'd like to discuss its implementation. The suggested
> approach is to rewrite given query turning it from few INSERTs into
> single statement and processing arguments accordingly. I suggest this
> as long as the whole point of batching is to make as little
> interactions with cluster as possible and to make operations as
> condensed as possible, and in case of Ignite it means that we should
> send as little JdbcQueryTasks as possible. And, as long as a query
> task holds single query and its arguments, this approach will not
> require any changes to be done to current design and won't break any
> backward compatibility - all dirty work on rewriting will be done by
> JDBC driver.
> Without rewriting, we could introduce some new query task for batch
> operations, but that would make impossible sending such requests from
> newer clients to older servers (say, servers of version 1.8.0, which
> does not know about batching, let alone older versions).
> I'd like to hear comments and suggestions from the community. Thanks!
>
> - Alex
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message