ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Setrakyan <dsetrak...@apache.org>
Subject Re: Ignite 2.0 tasks/roadmap
Date Thu, 14 Jul 2016 22:53:19 GMT
On Fri, Jul 15, 2016 at 1:49 AM, Alexey Goncharuk <
alexey.goncharuk@gmail.com> wrote:

> >
> > >
> > > I know there is IgniteDataStreamer for writing cache, but how about
> > > reading cache as stream for iterate all elements with scan performane
> > 1-3M
> > > tuple/sec?
> > >
> >
> > We already have Scan queries which allow for paginated iteration with
> > filters. Are you suggesting something beyond this?
>
>
> I like the idea of DataStreamer approach for scanning a cache. I think it
> would be nice to have a way to iterate over cache partitions in parallel,
> similar to forEachPartition() method in Spark RDD.
>
> Benefits compared to current Scan query:
>  * Parallel execution for different partitions
>  * Bringing computation to data, not data to client.
>
> Of course, this can already be implemented by a user with local scan query
> + compute task, but having an utility method on an API will cut a lot of
> boilerplate code for users.
>

Got it now. Sounds very useful. I think we should definitely create a
ticket for it and see if anyone in the community will pick it up. Sounds
like it won’t be too difficult to implement.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message