ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexei Scherbakov <alexey.scherbak...@gmail.com>
Subject Re: How to call loadCache before node is started ?
Date Sat, 18 Jun 2016 11:02:29 GMT
Hi, Kristian.

I haven't heard about anything like atomic loading of cache.

You should be safe calling loadCache just after getOrCreate.

Every node will start loading data, skipping keys which are not belong to
them.

There is no such thing as partial replication, data is always consistent.

To improve efficiency of initial load you may use partition aware loading
as described here [1]

Another approach may be using DataStreamer on dedicated data loading node
and marking some
state (on example field in database) when loading is completed.

Cache users must wait until the field is set to true.

You can also use Ignite's distributed locking for that purpose [2]

[1]
https://apacheignite.readme.io/docs/data-loading#section-partition-aware-data-loading
[2] https://apacheignite.readme.io/docs/distributed-locks


2016-06-18 11:55 GMT+03:00 Kristian Rosenvold <krosenvold@apache.org>:

> Our current node startup logic includes a simple heuristic like this:
>
> final IgniteCache cache = ignite.getOrCreateCache(configuration);
> if (cache.localSize(CachePeekMode.ALL) == 0) {
>    LOGGER.info("Empty cache or No-one else around in the ignite cloud,
> loading cache {} from database", configuration.getName() );
>    cache.loadCache((k, v) -> true, Integer.MAX_VALUE);
> }
>
> The problem with this logic is that that cache is started after
> getOrCreateCache, and other cluster members coming up while the first
> node is starting can get partial replication. How can I make loadCache
> happen during getOrCreateCache to ensure atomic startup/loading of the
> cache ?
>
> Kristian
>



-- 

Best regards,
Alexei Scherbakov

Mime
View raw message