ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pavel Tupitsyn <ptupit...@apache.org>
Subject Re: DML data streaming
Date Wed, 15 Feb 2017 12:56:07 GMT
I have no particular opinion on how we should handle _key/_val,
but we certainly need a way to select entire key and value objects via
SqlFieldsQuery,
and this should work without any additional configuration.

We can rename these, turn them into system functions, whatever.

Ignite.NET LINQ provider heavily relies on this possibility - users often
want to select the entire entry value.

On Wed, Feb 15, 2017 at 3:28 PM, Vladimir Ozerov <vozerov@gridgain.com>
wrote:

> Ok, let's put aside current fields configuration, I'll create separate
> thread for it. As far as _KEY and _VAL, proposed change is exactly about
> mappings:
>
> class QueryEntity {
>     ...
>     String keyFieldName;
>     String valFieldName;
>     ...
> }
>
> The key thing is that we will not require users to be aware of our system
> columns. Normally user should not bother about existence of hidden _KEY and
> _VAL columns. Instead, we just allow them to optionally reference the whole
> key and/or val through predefined name.
>
> On Wed, Feb 15, 2017 at 3:07 PM, Sergi Vladykin <sergi.vladykin@gmail.com>
> wrote:
>
> > I don't see any improvement here. Usability will only suffer with this
> > change.
> >
> > I'd suggest to just add mapping for system columns like _key, _val ,
> _ver.
> >
> > Sergi
> >
> > 2017-02-15 13:18 GMT+03:00 Vladimir Ozerov <vozerov@gridgain.com>:
> >
> > > I think the whole QueryEntity class require rework to allow for this
> > > change. I would start with creating QueryField class which will
> > encapsulate
> > > all field properties which are currently set through different setters:
> > >
> > > class QueryField {
> > >     String name;
> > >     String type;
> > >     String alias;
> > >     boolean keyField;
> > > }
> > >
> > > class QueryEntity {
> > >     String tableName;
> > >     String keyType;
> > >     String valType;
> > >     Collection<QueryField> fields;
> > >     Collection<QueryIndex> indexes;
> > > }
> > >
> > > Then we can add optional key and value field names to top-level config.
> > If
> > > set, key and/or value will have names and will be included into SELECT
> *
> > > query in the same way as we do this for _KEY and _VAL at the moment:
> > >
> > > class QueryEntity {
> > >     String tableName;
> > >     String keyType;
> > >     String valType;
> > > *    String keyFieldName;*
> > > *    String valFieldName;*
> > >     Collection<QueryField> fields;
> > >     Collection<QueryIndex> indexes;
> > > }
> > >
> > > Any other ideas?
> > >
> > > On Tue, Feb 14, 2017 at 9:19 PM, Dmitriy Setrakyan <
> > dsetrakyan@apache.org>
> > > wrote:
> > >
> > > > Vova,
> > > >
> > > > Agree about the primitive types. However, it is not clear to me how
> the
> > > > mapping from a primitive type to a column name will be supported. Do
> > you
> > > > have a design in mind?
> > > >
> > > > D.
> > > >
> > > > On Tue, Feb 14, 2017 at 6:16 AM, Vladimir Ozerov <
> vozerov@gridgain.com
> > >
> > > > wrote:
> > > >
> > > > > Dima,
> > > > >
> > > > > This will not work for primitive keys and values as currently the
> > only
> > > > way
> > > > > to address them is to use "_KEY" and "_VAL" aliases respectively.
> For
> > > > this
> > > > > reason I would rather postpone UPDATE/DELETE implementation until
> > > "_KEY"
> > > > > and "_VAL" are hidden from public API and some kind of mapping is
> > > > > introduced. AFAIK this should be handled as a part of IGNITE-3487
> > ]1].
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-3487
> > > > >
> > > > > On Sat, Feb 11, 2017 at 3:36 AM, Dmitriy Setrakyan <
> > > > dsetrakyan@apache.org>
> > > > > wrote:
> > > > >
> > > > > > On Fri, Feb 10, 2017 at 3:36 AM, Vladimir Ozerov <
> > > vozerov@gridgain.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > I propose to ship streaming with INSERT support only for
now.
> > This
> > > is
> > > > > > > enough for multitude cases and will add value to Ignite
1.9
> > > > > immediately.
> > > > > > We
> > > > > > > can think about correct streaming UPDATE/DELETE architecture
> > > > separately
> > > > > > .It
> > > > > > > is much more difficult thing, we cannot support it in a
clean
> way
> > > > right
> > > > > > now
> > > > > > > due to multiple "_key" and "_val" usages over the code
base.
> > > > > > >
> > > > > >
> > > > > > Vova, I disagree. If all parts of the key are present, then
we
> can
> > > > always
> > > > > > construct a key in all cases. For these operations we can always
> > > > support
> > > > > > streaming. For all other operations, we can delegate to standard
> > MR,
> > > > but
> > > > > > still perform most operations on the same node, as I suggested
in
> > > > another
> > > > > > email.
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message