ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergi Vladykin <sergi.vlady...@gmail.com>
Subject Re: DML data streaming
Date Wed, 15 Feb 2017 13:03:28 GMT
Vladimir,

Looks good to me.


Pavel,

No worries, it will work exactly like you described: hidden _key and _val
fields will be always accessible.

Sergi

2017-02-15 15:56 GMT+03:00 Pavel Tupitsyn <ptupitsyn@apache.org>:

> I have no particular opinion on how we should handle _key/_val,
> but we certainly need a way to select entire key and value objects via
> SqlFieldsQuery,
> and this should work without any additional configuration.
>
> We can rename these, turn them into system functions, whatever.
>
> Ignite.NET LINQ provider heavily relies on this possibility - users often
> want to select the entire entry value.
>
> On Wed, Feb 15, 2017 at 3:28 PM, Vladimir Ozerov <vozerov@gridgain.com>
> wrote:
>
> > Ok, let's put aside current fields configuration, I'll create separate
> > thread for it. As far as _KEY and _VAL, proposed change is exactly about
> > mappings:
> >
> > class QueryEntity {
> >     ...
> >     String keyFieldName;
> >     String valFieldName;
> >     ...
> > }
> >
> > The key thing is that we will not require users to be aware of our system
> > columns. Normally user should not bother about existence of hidden _KEY
> and
> > _VAL columns. Instead, we just allow them to optionally reference the
> whole
> > key and/or val through predefined name.
> >
> > On Wed, Feb 15, 2017 at 3:07 PM, Sergi Vladykin <
> sergi.vladykin@gmail.com>
> > wrote:
> >
> > > I don't see any improvement here. Usability will only suffer with this
> > > change.
> > >
> > > I'd suggest to just add mapping for system columns like _key, _val ,
> > _ver.
> > >
> > > Sergi
> > >
> > > 2017-02-15 13:18 GMT+03:00 Vladimir Ozerov <vozerov@gridgain.com>:
> > >
> > > > I think the whole QueryEntity class require rework to allow for this
> > > > change. I would start with creating QueryField class which will
> > > encapsulate
> > > > all field properties which are currently set through different
> setters:
> > > >
> > > > class QueryField {
> > > >     String name;
> > > >     String type;
> > > >     String alias;
> > > >     boolean keyField;
> > > > }
> > > >
> > > > class QueryEntity {
> > > >     String tableName;
> > > >     String keyType;
> > > >     String valType;
> > > >     Collection<QueryField> fields;
> > > >     Collection<QueryIndex> indexes;
> > > > }
> > > >
> > > > Then we can add optional key and value field names to top-level
> config.
> > > If
> > > > set, key and/or value will have names and will be included into
> SELECT
> > *
> > > > query in the same way as we do this for _KEY and _VAL at the moment:
> > > >
> > > > class QueryEntity {
> > > >     String tableName;
> > > >     String keyType;
> > > >     String valType;
> > > > *    String keyFieldName;*
> > > > *    String valFieldName;*
> > > >     Collection<QueryField> fields;
> > > >     Collection<QueryIndex> indexes;
> > > > }
> > > >
> > > > Any other ideas?
> > > >
> > > > On Tue, Feb 14, 2017 at 9:19 PM, Dmitriy Setrakyan <
> > > dsetrakyan@apache.org>
> > > > wrote:
> > > >
> > > > > Vova,
> > > > >
> > > > > Agree about the primitive types. However, it is not clear to me how
> > the
> > > > > mapping from a primitive type to a column name will be supported.
> Do
> > > you
> > > > > have a design in mind?
> > > > >
> > > > > D.
> > > > >
> > > > > On Tue, Feb 14, 2017 at 6:16 AM, Vladimir Ozerov <
> > vozerov@gridgain.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Dima,
> > > > > >
> > > > > > This will not work for primitive keys and values as currently
the
> > > only
> > > > > way
> > > > > > to address them is to use "_KEY" and "_VAL" aliases respectively.
> > For
> > > > > this
> > > > > > reason I would rather postpone UPDATE/DELETE implementation
until
> > > > "_KEY"
> > > > > > and "_VAL" are hidden from public API and some kind of mapping
is
> > > > > > introduced. AFAIK this should be handled as a part of IGNITE-3487
> > > ]1].
> > > > > >
> > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-3487
> > > > > >
> > > > > > On Sat, Feb 11, 2017 at 3:36 AM, Dmitriy Setrakyan <
> > > > > dsetrakyan@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > > > On Fri, Feb 10, 2017 at 3:36 AM, Vladimir Ozerov <
> > > > vozerov@gridgain.com
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > I propose to ship streaming with INSERT support only
for now.
> > > This
> > > > is
> > > > > > > > enough for multitude cases and will add value to Ignite
1.9
> > > > > > immediately.
> > > > > > > We
> > > > > > > > can think about correct streaming UPDATE/DELETE architecture
> > > > > separately
> > > > > > > .It
> > > > > > > > is much more difficult thing, we cannot support it
in a clean
> > way
> > > > > right
> > > > > > > now
> > > > > > > > due to multiple "_key" and "_val" usages over the
code base.
> > > > > > > >
> > > > > > >
> > > > > > > Vova, I disagree. If all parts of the key are present,
then we
> > can
> > > > > always
> > > > > > > construct a key in all cases. For these operations we can
> always
> > > > > support
> > > > > > > streaming. For all other operations, we can delegate to
> standard
> > > MR,
> > > > > but
> > > > > > > still perform most operations on the same node, as I suggested
> in
> > > > > another
> > > > > > > email.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message