ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Setrakyan <dsetrak...@apache.org>
Subject Re: DML data streaming
Date Thu, 16 Feb 2017 03:24:44 GMT
On Wed, Feb 15, 2017 at 2:41 PM, Alexander Paschenko <
alexander.a.paschenko@gmail.com> wrote:

> Folks,
>
> Regarding INSERT semantics in JDBC DML streaming mode - I've left only
> INSERTs supports as we'd agreed before.
>
> However, current architecture of streaming related internals does not
> give any clear way to intercept key duplicates and inform the user -
> say, I can't just throw an exception from stream receiver (which is to
> my knowledge the only place where we could filter erroneous keys) as
> long as it will lead to whole batch remap and it's clearly not what we
> want here.
>
> Printing warning to log from the receiver is of little to no use as it
> will happen on data nodes so the end user won't see anything.
>

However, you still must do it. You should try throttling the identical log
messages, so we don't flood the log.


>
> What I've introduced for now is optional config param that turns on
> allowOverwrite on the streamer used in DML operation.
>

Agree, sounds like a good use of the flag. Are you setting it via JDBC/ODBC
connection flag?


> Does anyone have any thoughts about what could/should be done
> regarding informing user about key duplicates in streaming mode? Or
> probably we should just let it be as it is now?
>

In my view, we should introduce some generic error trap callback, e.g.
onSqlError(...), for all unhandled SQL errors. User should provide it in
the configuration, before startup. What do you think?


>
> Regards,
> Alex
>
> 2017-02-15 23:42 GMT+03:00 Dmitriy Setrakyan <dsetrakyan@apache.org>:
> > On Wed, Feb 15, 2017 at 4:28 AM, Vladimir Ozerov <vozerov@gridgain.com>
> > wrote:
> >
> >> Ok, let's put aside current fields configuration, I'll create separate
> >> thread for it. As far as _KEY and _VAL, proposed change is exactly about
> >> mappings:
> >>
> >> class QueryEntity {
> >>     ...
> >>     String keyFieldName;
> >>     String valFieldName;
> >>     ...
> >> }
> >>
> >> The key thing is that we will not require users to be aware of our
> system
> >> columns. Normally user should not bother about existence of hidden _KEY
> and
> >> _VAL columns. Instead, we just allow them to optionally reference the
> whole
> >> key and/or val through predefined name.
> >>
> >>
> > Vladimir, how will it work from the DDL perspective. Let's say whenever
> > user wants to create a table in Ignite?
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message