ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrey Kornev <andrewkor...@hotmail.com>
Subject Re: Transformers in SCAN queries
Date Thu, 04 Feb 2016 08:13:42 GMT
Another perhaps bigger problem with running queries (including scan queries) using closures
was discussed at length on the @dev not so long ago. It has to do with partitions migration
due to cluster topology changes which may result in the query returning incomplete result.
And while it is possible to solve this problem for the scan queries by using some clever tricks,
all bets are off with the SQL queries.Andrey
    _____________________________
From: Valentin Kulichenko <valentin.kulichenko@gmail.com>
Sent: Thursday, February 4, 2016 6:29 AM
Subject: Re: Transformers in SCAN queries
To:  <dev@ignite.apache.org>


                   Dmitry,   
    
 The main difference in my view is that you lose pagination when sending   
 results from servers to client. What if one wants to iterate through all   
 entries in cache?   
    
 On Wed, Feb 3, 2016 at 9:47 PM, Dmitriy Setrakyan <dsetrakyan@apache.org>   
 wrote:   
    
 > Valentin,   
 >   
 > Wouldn’t the same effect be achieved by broadcasting a closure to the   
 > cluster and executing scan-query on every node locally?   
 >   
 > D.   
 >   
 > On Wed, Feb 3, 2016 at 9:17 PM, Valentin Kulichenko <   
 > valentin.kulichenko@gmail.com> wrote:   
 >   
 > > Igniters,   
 > >   
 > > I keep getting requests from our users to add optional transformers to   
 > SCAN   
 > > queries. This will allow to iterate through cache, but do not transfer   
 > > whole key-value pairs across networks (e.g., get only keys). The feature   
 > > looks useful and I created a ticket [1].   
 > >   
 > > I am struggling with the design now. The problem is that I wanted to   
 > extend   
 > > existing ScanQuery object for this, but this seems to be impossible   
 > because   
 > > it already extends Query<Cache.Entry<K, V>> and thus can iterate only
  
 > > through entries.   
 > >   
 > > The only option I see now is to create a separate query type, copy-paste   
 > > everything from ScanQuery and add *mandatory* transformer. Something like   
 > > this:   
 > >   
 > > ScanTransformQuery<K, V, R> extends Query<R> {   
 > >     IgniteBiPredicate<K, V> filter;   
 > >     IgniteClosure<Cache.Entry<K, V>, R> transformer;   
 > >     int part;   
 > >     ...   
 > > }   
 > >   
 > > Thoughts? Does anyone has other ideas?   
 > >   
 > > [1]    https://issues.apache.org/jira/browse/IGNITE-2546   
 > >   
 > > -Val   
 > >   
 >   
       


  
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message