ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrey Gura <ag...@gridgain.com>
Subject Re: Transformers in SCAN queries
Date Thu, 04 Feb 2016 17:10:17 GMT
SqlQuery, TextQuery and SpiQuery are similar to ScanQuery because all of
them also defined as Query<Cache.Entry<>>.

It can be usefull to have one query SqlQuery that can return different
result that will be produced from cache entry by transformer.

Actually only SqlFieldsQuery has different definition. So transformers can
be applied to any type of query (including SqlFieldsQuery, I believe).

On Thu, Feb 4, 2016 at 7:42 PM, Sergi Vladykin <sergi.vladykin@gmail.com>
wrote:

> I don't like the idea of having additional method *query(Query<E> qry,
> Transformer<E, R> transfomer); *because I don't see how these transformers
> will work for example with SQL, but this API makes you think that
> transformers are supported for all the query types.
>
> Sergi
>
> 2016-02-04 16:46 GMT+03:00 Andrey Gura <agura@gridgain.com>:
>
> > Val,
> >
> > can we introduce new method into IgnteCache API?
> >
> > Now we have method: public <R> QueryCursor<R> query(Query<R> qry);
> >
> > New method will be something like this: <R> QueryCursor<R> query(Query<E>
> > qry, Transformer<E, R> transfomer);
> >
> > It allows provide transformers for all query types and chnages will be
> > related only with query cursor functionality.
> >
> > Will it work?
> >
> > On Thu, Feb 4, 2016 at 11:13 AM, Andrey Kornev <andrewkornev@hotmail.com
> >
> > wrote:
> >
> > > Another perhaps bigger problem with running queries (including scan
> > > queries) using closures was discussed at length on the @dev not so long
> > > ago. It has to do with partitions migration due to cluster topology
> > changes
> > > which may result in the query returning incomplete result. And while it
> > is
> > > possible to solve this problem for the scan queries by using some
> clever
> > > tricks, all bets are off with the SQL queries.Andrey
> > >     _____________________________
> > > From: Valentin Kulichenko <valentin.kulichenko@gmail.com>
> > > Sent: Thursday, February 4, 2016 6:29 AM
> > > Subject: Re: Transformers in SCAN queries
> > > To:  <dev@ignite.apache.org>
> > >
> > >
> > >                    Dmitry,
> > >
> > >  The main difference in my view is that you lose pagination when
> sending
> > >  results from servers to client. What if one wants to iterate through
> all
> > >  entries in cache?
> > >
> > >  On Wed, Feb 3, 2016 at 9:47 PM, Dmitriy Setrakyan <
> > dsetrakyan@apache.org>
> > >  wrote:
> > >
> > >  > Valentin,
> > >  >
> > >  > Wouldn’t the same effect be achieved by broadcasting a closure to
> the
> > >  > cluster and executing scan-query on every node locally?
> > >  >
> > >  > D.
> > >  >
> > >  > On Wed, Feb 3, 2016 at 9:17 PM, Valentin Kulichenko <
> > >  > valentin.kulichenko@gmail.com> wrote:
> > >  >
> > >  > > Igniters,
> > >  > >
> > >  > > I keep getting requests from our users to add optional
> transformers
> > to
> > >  > SCAN
> > >  > > queries. This will allow to iterate through cache, but do not
> > transfer
> > >  > > whole key-value pairs across networks (e.g., get only keys). The
> > > feature
> > >  > > looks useful and I created a ticket [1].
> > >  > >
> > >  > > I am struggling with the design now. The problem is that I wanted
> to
> > >  > extend
> > >  > > existing ScanQuery object for this, but this seems to be
> impossible
> > >  > because
> > >  > > it already extends Query<Cache.Entry<K, V>> and thus
can iterate
> > only
> > >  > > through entries.
> > >  > >
> > >  > > The only option I see now is to create a separate query type,
> > > copy-paste
> > >  > > everything from ScanQuery and add *mandatory* transformer.
> Something
> > > like
> > >  > > this:
> > >  > >
> > >  > > ScanTransformQuery<K, V, R> extends Query<R> {
> > >  > >     IgniteBiPredicate<K, V> filter;
> > >  > >     IgniteClosure<Cache.Entry<K, V>, R> transformer;
> > >  > >     int part;
> > >  > >     ...
> > >  > > }
> > >  > >
> > >  > > Thoughts? Does anyone has other ideas?
> > >  > >
> > >  > > [1]    https://issues.apache.org/jira/browse/IGNITE-2546
> > >  > >
> > >  > > -Val
> > >  > >
> > >  >
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
> > --
> > Andrey Gura
> > GridGain Systems, Inc.
> > www.gridgain.com
> >
>



-- 
Andrey Gura
GridGain Systems, Inc.
www.gridgain.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message