ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexey Goncharuk <alexey.goncha...@gmail.com>
Subject Re: Ignite 2.0 tasks/roadmap
Date Thu, 14 Jul 2016 22:49:19 GMT
> >
> > I know there is IgniteDataStreamer for writing cache, but how about
> > reading cache as stream for iterate all elements with scan performane
> 1-3M
> > tuple/sec?
> >
> We already have Scan queries which allow for paginated iteration with
> filters. Are you suggesting something beyond this?

I like the idea of DataStreamer approach for scanning a cache. I think it
would be nice to have a way to iterate over cache partitions in parallel,
similar to forEachPartition() method in Spark RDD.

Benefits compared to current Scan query:
 * Parallel execution for different partitions
 * Bringing computation to data, not data to client.

Of course, this can already be implemented by a user with local scan query
+ compute task, but having an utility method on an API will cut a lot of
boilerplate code for users.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message