ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From abarbaro <abarb...@powercosts.com>
Subject Re: Ignite Cache and Compute Performance
Date Wed, 30 Mar 2016 13:41:43 GMT
Ok, I will send examples later on today with performance comparisons between
iterating and doing ScanQuery.

Regarding the ability to query partitions, it would be fine to use ScanQuery
(assuming the performance is not an issue); however, I still see a possible
enhancement to that API to be able to pass a Set<Integer> or int[]
partitions parameter.  That way it can get data for multiple partitions in
one shot.  I think this is a very common scenario found in map-reduce

Basically, if you iterate/scan one partition at a time you'll have more
overhead than scanning in batches of partitions. For example, Ignite has
1024 partitions on a cache by default, and if I were to submit 1024
map-reduce jobs to each node for the map-reduce computation I would be a lot
slower than submitting say 8 jobs each of them scanning 128 partitions.


View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Cache-and-Compute-Performance-tp3722p3783.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

View raw message