ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Ozerov <voze...@gridgain.com>
Subject Re: Ability to disable WAL in runtime to speed up load
Date Wed, 25 Oct 2017 11:55:51 GMT
Pavel,

This feature will work independently of streamer. If you want to load data
with streamer, then you disable WAL first through some API call or SQL
command, and then start loading.

On Wed, Oct 25, 2017 at 2:41 PM, Pavel Tupitsyn <ptupitsyn@apache.org>
wrote:

> IMO IgniteCache.disableWal() should be enough.
>
> Also what about an option to disable WAL when IgniteDataStreamer is active?
>
> On Wed, Oct 25, 2017 at 2:38 PM, Vladimir Ozerov <vozerov@gridgain.com>
> wrote:
>
> > One more idea - ability to create a cache with initially disabled WAL.
> > Might be useful.
> >
> > On Wed, Oct 25, 2017 at 2:35 PM, Vladimir Ozerov <vozerov@gridgain.com>
> > wrote:
> >
> > > Igniters,
> > >
> > > We have a ticket to implement runtime WAL state management [1].It will
> be
> > > possible to disable it temporarily. This is very useful for data
> loading
> > > case. Our experiments show that we can improve data loading time by a
> > > factor of 2x-10x depending on configuration, indexes and cluster
> > topology.
> > >
> > > We had a private chat with Anton Vinogradov and Alexey Goncharuk, and
> > came
> > > to certain design which you can find inside the ticket. Comments are
> > > welcomed.
> > >
> > > In this email I would like to focus on API of this feature.
> > Considerations:
> > > 1) It should be possible to disable WAL globally or on per-cache basis.
> > > 2) It should be possible to get current state of WAL for the cache.
> > >
> > > Quick draft from my side:
> > > void disableWal();
> > > void disableWal(String... cacheNames);
> > > void enableWal();
> > > void enableWal(String... cacheNames);
> > > boolean isWalEnabled(String... cacheNames);
> > >
> > > Please help improving it.
> > > - Do we really need methods to disable WAL for all caches? This is not
> > > very intuitive feature.
> > > - Where to place these operations? Ignite interface? Ignite +
> > IgniteCache?
> > > Separate facade?
> > > - Do we need async counterparts?
> > > - Do we need a feature which completes when WAL is enabled back?
> > >
> > > Vladimir.
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-6411
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message