ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrey Gura <ag...@gridgain.com>
Subject Re: Transformers in SCAN queries
Date Thu, 04 Feb 2016 18:08:19 GMT
Sergi,


> What you are going to transform remotely here?


I'm not going. I believe :)

Just hypothetical use case: You have one SqlFieldsQuery but different
requirements for returned values. For one case you have to return some
string fields as is and for another case you have to trim string to 32
characters. Of course you still can trim strings locally but transformers
allow you do it remotely.

Anyway, this solution can be usefull for rest query types.

On Thu, Feb 4, 2016 at 8:54 PM, Sergi Vladykin <sergi.vladykin@gmail.com>
wrote:

> The whole point of Transformer is to do remote transform, how will this
> work with SqlFieldsQuery? What you are going to transform remotely here? I
> believe all the transformations must happen at SQL level here.
>
> Sergi
>
>
>
> 2016-02-04 20:10 GMT+03:00 Andrey Gura <agura@gridgain.com>:
>
> > SqlQuery, TextQuery and SpiQuery are similar to ScanQuery because all of
> > them also defined as Query<Cache.Entry<>>.
> >
> > It can be usefull to have one query SqlQuery that can return different
> > result that will be produced from cache entry by transformer.
> >
> > Actually only SqlFieldsQuery has different definition. So transformers
> can
> > be applied to any type of query (including SqlFieldsQuery, I believe).
> >
> > On Thu, Feb 4, 2016 at 7:42 PM, Sergi Vladykin <sergi.vladykin@gmail.com
> >
> > wrote:
> >
> > > I don't like the idea of having additional method *query(Query<E> qry,
> > > Transformer<E, R> transfomer); *because I don't see how these
> > transformers
> > > will work for example with SQL, but this API makes you think that
> > > transformers are supported for all the query types.
> > >
> > > Sergi
> > >
> > > 2016-02-04 16:46 GMT+03:00 Andrey Gura <agura@gridgain.com>:
> > >
> > > > Val,
> > > >
> > > > can we introduce new method into IgnteCache API?
> > > >
> > > > Now we have method: public <R> QueryCursor<R> query(Query<R>
qry);
> > > >
> > > > New method will be something like this: <R> QueryCursor<R>
> > query(Query<E>
> > > > qry, Transformer<E, R> transfomer);
> > > >
> > > > It allows provide transformers for all query types and chnages will
> be
> > > > related only with query cursor functionality.
> > > >
> > > > Will it work?
> > > >
> > > > On Thu, Feb 4, 2016 at 11:13 AM, Andrey Kornev <
> > andrewkornev@hotmail.com
> > > >
> > > > wrote:
> > > >
> > > > > Another perhaps bigger problem with running queries (including scan
> > > > > queries) using closures was discussed at length on the @dev not so
> > long
> > > > > ago. It has to do with partitions migration due to cluster topology
> > > > changes
> > > > > which may result in the query returning incomplete result. And
> while
> > it
> > > > is
> > > > > possible to solve this problem for the scan queries by using some
> > > clever
> > > > > tricks, all bets are off with the SQL queries.Andrey
> > > > >     _____________________________
> > > > > From: Valentin Kulichenko <valentin.kulichenko@gmail.com>
> > > > > Sent: Thursday, February 4, 2016 6:29 AM
> > > > > Subject: Re: Transformers in SCAN queries
> > > > > To:  <dev@ignite.apache.org>
> > > > >
> > > > >
> > > > >                    Dmitry,
> > > > >
> > > > >  The main difference in my view is that you lose pagination when
> > > sending
> > > > >  results from servers to client. What if one wants to iterate
> through
> > > all
> > > > >  entries in cache?
> > > > >
> > > > >  On Wed, Feb 3, 2016 at 9:47 PM, Dmitriy Setrakyan <
> > > > dsetrakyan@apache.org>
> > > > >  wrote:
> > > > >
> > > > >  > Valentin,
> > > > >  >
> > > > >  > Wouldn’t the same effect be achieved by broadcasting a closure
> to
> > > the
> > > > >  > cluster and executing scan-query on every node locally?
> > > > >  >
> > > > >  > D.
> > > > >  >
> > > > >  > On Wed, Feb 3, 2016 at 9:17 PM, Valentin Kulichenko <
> > > > >  > valentin.kulichenko@gmail.com> wrote:
> > > > >  >
> > > > >  > > Igniters,
> > > > >  > >
> > > > >  > > I keep getting requests from our users to add optional
> > > transformers
> > > > to
> > > > >  > SCAN
> > > > >  > > queries. This will allow to iterate through cache, but
do not
> > > > transfer
> > > > >  > > whole key-value pairs across networks (e.g., get only
keys).
> The
> > > > > feature
> > > > >  > > looks useful and I created a ticket [1].
> > > > >  > >
> > > > >  > > I am struggling with the design now. The problem is that
I
> > wanted
> > > to
> > > > >  > extend
> > > > >  > > existing ScanQuery object for this, but this seems to
be
> > > impossible
> > > > >  > because
> > > > >  > > it already extends Query<Cache.Entry<K, V>>
and thus can
> iterate
> > > > only
> > > > >  > > through entries.
> > > > >  > >
> > > > >  > > The only option I see now is to create a separate query
type,
> > > > > copy-paste
> > > > >  > > everything from ScanQuery and add *mandatory* transformer.
> > > Something
> > > > > like
> > > > >  > > this:
> > > > >  > >
> > > > >  > > ScanTransformQuery<K, V, R> extends Query<R>
{
> > > > >  > >     IgniteBiPredicate<K, V> filter;
> > > > >  > >     IgniteClosure<Cache.Entry<K, V>, R> transformer;
> > > > >  > >     int part;
> > > > >  > >     ...
> > > > >  > > }
> > > > >  > >
> > > > >  > > Thoughts? Does anyone has other ideas?
> > > > >  > >
> > > > >  > > [1]    https://issues.apache.org/jira/browse/IGNITE-2546
> > > > >  > >
> > > > >  > > -Val
> > > > >  > >
> > > > >  >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Andrey Gura
> > > > GridGain Systems, Inc.
> > > > www.gridgain.com
> > > >
> > >
> >
> >
> >
> > --
> > Andrey Gura
> > GridGain Systems, Inc.
> > www.gridgain.com
> >
>



-- 
Andrey Gura
GridGain Systems, Inc.
www.gridgain.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message