ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@gridgain.com>
Subject Re: Data Loading Performance Issue
Date Mon, 16 Nov 2015 07:44:09 GMT

Do you use the DB mentioned below as a persistent storage for the Ignite Cache being populated?
If yes, then you can leverage CacheStore interface [1].

If otherwise you migrate your data from the DB then IgniteDataStreamer is the best candidate
for such kind of tasks.
It’s not clear from your description below how you actually use the APIs, what the configuration
is, how big your data set.
Please provide this info. 

[1] https://apacheignite.readme.io/docs/persistent-store#cachestore


> On 14 нояб. 2015 г., at 3:35, diopek <deha.peker@gmail.com> wrote:
> I am having performance issue for loading data into Ignite cache. I tried
> several ways, 
> from individual put to putAll to IgniteDataStreamer.addData. As one unit
> test case example; 
> - time to retrieve data from data base is 26 secs by populating HashMap.
> - if I try doing cache.putAll or dataStreamer.addData to bulk load data into
> Ignite cache takes another 240 secs. And one large scale batch run cache
> population took 5 hrs which is quite extreme as actual batch took less than
> 2hrs after that. Wondering if I am missing some fundementals here. Please
> advise.
> --
> View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-Loading-Performance-Issue-tp1958.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.

View raw message