ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Paschenko <alexander.a.pasche...@gmail.com>
Subject Re: Batch DML queries design discussion
Date Thu, 08 Dec 2016 12:04:42 GMT
Vlad,

1. Of course API to my view should be well-known JDBC one - addBatch
and friends. Question was not about API but rather about
implementation. What do you mean by "copy locally and execute all at
once"?

2. As I see it, it does not contradict the 1st approach and could be
implemented on par as well.

Thanks,
Alex

2016-12-08 14:16 GMT+03:00 Vladimir Ozerov <vozerov@gridgain.com>:
> Hi Alex,
>
> To my understanding there are two possible approaches to batching in JDBC
> layer:
>
> 1) Rely on default batching API. Specifically
> *PreparedStatement.addBatch()* [1]
> and others. This is nice and clear API, users are used to it, and it's
> adoption will minimize user code changes when migrating from other JDBC
> sources. We simply copy updates locally and then execute them all at once
> with only a single network hop to servers. *IgniteDataStreamer* can be used
> underneath.
>
> 2) Or we can have separate connection flag which will move all
> INSERT/UPDATE/DELETE statements through streamer.
>
> I prefer the first approach
>
> Also we need to keep in mind that data streamer has poor performance when
> adding single key-value pairs due to high overhead on concurrency and other
> bookkeeping. Instead, it is better to pre-batch key-value pairs before
> giving them to streamer.
>
> Vladimir.
>
> [1]
> https://docs.oracle.com/javase/8/docs/api/java/sql/PreparedStatement.html#addBatch--
>
> On Thu, Dec 8, 2016 at 1:21 PM, Alexander Paschenko <
> alexander.a.paschenko@gmail.com> wrote:
>
>> Hello Igniters,
>>
>> One of the major improvements to DML has to be support of batch
>> statements. I'd like to discuss its implementation. The suggested
>> approach is to rewrite given query turning it from few INSERTs into
>> single statement and processing arguments accordingly. I suggest this
>> as long as the whole point of batching is to make as little
>> interactions with cluster as possible and to make operations as
>> condensed as possible, and in case of Ignite it means that we should
>> send as little JdbcQueryTasks as possible. And, as long as a query
>> task holds single query and its arguments, this approach will not
>> require any changes to be done to current design and won't break any
>> backward compatibility - all dirty work on rewriting will be done by
>> JDBC driver.
>> Without rewriting, we could introduce some new query task for batch
>> operations, but that would make impossible sending such requests from
>> newer clients to older servers (say, servers of version 1.8.0, which
>> does not know about batching, let alone older versions).
>> I'd like to hear comments and suggestions from the community. Thanks!
>>
>> - Alex
>>

Mime
View raw message