ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nikolai Tikhonov <ntikho...@apache.org>
Subject Re: DataStreamer vs CacheStoreAdapter
Date Tue, 05 Dec 2017 16:59:29 GMT

You're thinking in the right way.

In the first case DataStreamer looks preferable. If I understood correctly
then you have a distributed database and you need to do some preparation
before loading data to Apache Ignite. In this case you can create needed
count of DataStreamer (one DataStreamer per cache) and start streaming data
via them. You can safe share the instances between threads. DataStreamer
API will be parallel loading itself, you don't need to worry about this.

The second case is not clear for me. Are going to split one table by
identifier to several caches? CacheStore should be used when your tables
directly mapped to Ignite Caches. For example you have a Person table and
you plan  going to move all data from it to one IgniteCache.

On Tue, Dec 5, 2017 at 7:30 PM, Lybrial <Lybrial@googlemail.com> wrote:

> Hello,
> im new to ignite and im not completely sure about the different use cases
> for `igeniteDataStreamer` and `CacheStoreAdapter`. In my application I have
> two different usecases but both rely on preloading data from a database
> into
> the ignite cache.
> 1. The first usecase is just some kind of bulk loading data into the cache.
> No transformations or additional work needed, just copy the whole database
> into the cache. I thought the `IgniteDataStreamer` would be the way to go
> here, is this correct? If yes: How could I optimize the data loading if my
> data is distributed between several databases. What If I have, lets say, 10
> databases with distinct data in each table but they have the same shema.
> Could I parallelize the dataloading so that all 10 databases are getting
> loaded into the cache at the same time.
> 2. The second usecase is that I load data from the database and group them
> into different caches meaning that for a given identifier (key) I will have
> a set of entries and every identifier will have its own cache. Does it make
> sense to use a `CacheStoreAdapter` here?
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/

View raw message