ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Ozerov <voze...@gridgain.com>
Subject Re: Ignite Cache and Compute Performance
Date Tue, 29 Mar 2016 11:49:17 GMT

I hardly can find this comparison fair. HashMap is plain non-thread safe
hashed map. Ignite is tread-safe, distributed cache with optional
transactions, evictions, expiration and so on. I would suggest you do the
following to improve numbers:
1) Set *CacheConfiguration.copyOnRead* flag to "false". This way you will
get rid of additional serializations which are not needed in your case.
2) Ensure you have enough heap so that GC doesn't stall your application
for a long time.
3) You use *IgniteCompute.broadcast()* which will execute the closure only
once, meaning that only one thread will perform calculation. This is not
common scenario for Ignite. Instead, it is better to use fully-fledged
map-reduce which will produce multiple jobs and utilize as much system
resources as possible. See *ComputeTaskMapExample*.
4) Instead of using iterator, it is better to use *ScanQuery* and it's
ability to scan particular cache partitions (method
*ScanQuery.setPartition()*). See *CacheQueryExample*.

Please let me know if you need any further assistance.


On Tue, Mar 29, 2016 at 12:33 AM, abarbaro <abarbaro@powercosts.com> wrote:

> Hello Ignite Community
> I'm trying to evaluate the performance of Ignite Cache and Distributed
> Computing.  To do so, I have set up a very simple example that is a
> simplification of the real-life examples for which we plan to use Ignite.
> To my surprise, the results show that Ignite is a significantly slower than
> I expected, and I would like to get some help with configuration and/or
> tuning to improve performance.
> Here's the example:  We have a warehouse that has 10,000,000 Inventory
> Items.  Each Item has a product Id, a quantity, and a price.  The sample
> computation is to calculate the Total Value of the inventory, which is
> SUM(i, Item[i].quantity * Item[i].price) (i.e. the sum of the value of each
> item).  The real-life is large-scale would have billions of entries per
> cache and more complex calculations.
> For the test, I have a single cache, with 10,000,000 entries, using pretty
> much Ignite's standard configuration with a few tweaks.  For only
> 10,000,000
> items, I would have expected this calculation to perform in a matter of few
> hundreds of milliseconds; however, the results for Ignite are almost 2
> order
> of magnitude slower are much slower (5,000 - 12,000 millis).  I have also a
> test that shows that when running this on the same JVM and using HashMap as
> the cache, the computation takes 92 millis.
> Also, the amount of memory overhead is very significant: the 10,000,000
> require almost 4 GB of heap on the Ignite node (single-node cluster),
> versus
> 1GB of heap when running within a JVM and using HashMap.
> Can you provide any guidance as to how to improve the performance of Ignite
> Cache + Compute for this simple case?  I'm attaching the code, and the
> corresponding configuration and settings.  I understand that there will be
> some overhead when using a distributed data grid as opposed to a HashMap on
> the same JVM, so I don't expect to get the same performance, but I would
> like to get as close as possible.
> Thanks so much in advance!
> *JVM Settings*
> JVM_OPTS=-Xms2048m -Xmx2048m -server -XX:+AggressiveOpts
> -XX:MaxPermSize=256m
> set JVM_OPTS=%JVM_OPTS% -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
> -XX:+UseTLAB -XX:NewSize=128m -XX:MaxNewSize=128m
> set JVM_OPTS=%JVM_OPTS% -XX:MaxTenuringThreshold=0 -XX:SurvivorRatio=1024
> -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60
> *Attachements*
> default-config.xml
> <
> http://apache-ignite-users.70518.x6.nabble.com/file/n3722/default-config.xml
> >
> <
> http://apache-ignite-users.70518.x6.nabble.com/file/n3722/Ignite_Performance_Analysis_1-Server_Node.png
> >
> Ignite_Performance_Analysis_Results.txt
> <
> http://apache-ignite-users.70518.x6.nabble.com/file/n3722/Ignite_Performance_Analysis_Results.txt
> >
> inventory_cache_compute.jar
> <
> http://apache-ignite-users.70518.x6.nabble.com/file/n3722/inventory_cache_compute.jar
> >
> CacheComputeTest.java
> <
> http://apache-ignite-users.70518.x6.nabble.com/file/n3722/CacheComputeTest.java
> >
> InventoryItem.java
> <
> http://apache-ignite-users.70518.x6.nabble.com/file/n3722/InventoryItem.java
> >
> InventoryValueCompute.java
> <
> http://apache-ignite-users.70518.x6.nabble.com/file/n3722/InventoryValueCompute.java
> >
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-Cache-and-Compute-Performance-tp3722.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.

View raw message