ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@apache.org>
Subject Re: Ability to disable WAL in runtime to speed up load
Date Wed, 25 Oct 2017 15:32:51 GMT
Guys,

Sever things to clarify:
- disabled WAL means disabled checkpointing (no data on disk at all)?
- after turning the WAL back when a preloading is over, how can I know a moment in time when
the disk holds the whole copy of data so that it’s safe to restart the whole cluster?

—
Denis

> On Oct 25, 2017, at 4:35 AM, Vladimir Ozerov <vozerov@gridgain.com> wrote:
> 
> Igniters,
> 
> We have a ticket to implement runtime WAL state management [1].It will be
> possible to disable it temporarily. This is very useful for data loading
> case. Our experiments show that we can improve data loading time by a
> factor of 2x-10x depending on configuration, indexes and cluster topology.
> 
> We had a private chat with Anton Vinogradov and Alexey Goncharuk, and came
> to certain design which you can find inside the ticket. Comments are
> welcomed.
> 
> In this email I would like to focus on API of this feature. Considerations:
> 1) It should be possible to disable WAL globally or on per-cache basis.
> 2) It should be possible to get current state of WAL for the cache.
> 
> Quick draft from my side:
> void disableWal();
> void disableWal(String... cacheNames);
> void enableWal();
> void enableWal(String... cacheNames);
> boolean isWalEnabled(String... cacheNames);
> 
> Please help improving it.
> - Do we really need methods to disable WAL for all caches? This is not very
> intuitive feature.
> - Where to place these operations? Ignite interface? Ignite + IgniteCache?
> Separate facade?
> - Do we need async counterparts?
> - Do we need a feature which completes when WAL is enabled back?
> 
> Vladimir.
> 
> [1] https://issues.apache.org/jira/browse/IGNITE-6411


Mime
View raw message