Mailing-List: contact user-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@ignite.apache.org
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 9.1 \(3096.5\))
Subject: Re: Data Loading Performance Issue
From: Denis Magda <dmagda@gridgain.com>
In-Reply-To: <1447461338394-1958.post@n6.nabble.com>
Date: Mon, 16 Nov 2015 10:44:09 +0300
Content-Transfer-Encoding: quoted-printable
Message-Id: <58C7309D-F3D1-4542-87C3-DF0092D08E2D@gridgain.com>
References: <1447461338394-1958.post@n6.nabble.com>
To: user@ignite.apache.org

Hi,

Do you use the DB mentioned below as a persistent storage for the Ignite =
Cache being populated?
If yes, then you can leverage CacheStore interface [1].

If otherwise you migrate your data from the DB then IgniteDataStreamer =
is the best candidate for such kind of tasks.
It=E2=80=99s not clear from your description below how you actually use =
the APIs, what the configuration is, how big your data set.
Please provide this info.=20

[1] https://apacheignite.readme.io/docs/persistent-store#cachestore

Regards,
Denis

> On 14 =D0=BD=D0=BE=D1=8F=D0=B1. 2015 =D0=B3., at 3:35, diopek =
<deha.peker@gmail.com> wrote:
>=20
> I am having performance issue for loading data into Ignite cache. I =
tried
> several ways,=20
> from individual put to putAll to IgniteDataStreamer.addData. As one =
unit
> test case example;=20
> - time to retrieve data from data base is 26 secs by populating =
HashMap.
> - if I try doing cache.putAll or dataStreamer.addData to bulk load =
data into
> Ignite cache takes another 240 secs. And one large scale batch run =
cache
> population took 5 hrs which is quite extreme as actual batch took less =
than
> 2hrs after that. Wondering if I am missing some fundementals here. =
Please
> advise.
>=20
>=20
>=20
> --
> View this message in context: =
http://apache-ignite-users.70518.x6.nabble.com/Data-Loading-Performance-Is=
sue-tp1958.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.