ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anton Vinogradov <avinogra...@gridgain.com>
Subject Re: Ability to disable WAL in runtime to speed up load
Date Wed, 25 Oct 2017 12:34:40 GMT
Pavel,

WAL disabling is a very dangerous operation and it seems to be not a good
idea to allow run regular operation with .disabledWal().
Let's think twice how to make new API safe.

On Wed, Oct 25, 2017 at 3:25 PM, Pavel Tupitsyn <ptupitsyn@apache.org>
wrote:

> Vladimir,
>
> It would be useful to be able to automatically disable WAL when streaming
> starts
> and re-enable after it ends, don't you think so?
>
> Something like IgniteDataStreamer.disableWal property.
>
> This is in addition to other API calls that you suggested.
>
> On Wed, Oct 25, 2017 at 3:25 PM, Alexey Goncharuk <
> alexey.goncharuk@gmail.com> wrote:
>
> > I do not like boolean isWalEnabled(String... cacheNames) - it's semantics
> > is confusing. Should it return true if WAL is enabled for all caches or
> if
> > WAL is enabled for at least one cache? IMO, since this is a local-read
> > operation, single cache per call is enough.
> >
> > As for the API placement, it looks like we need another facade
> > (IgniteControl ?) where active(bool) should also be moved. I do not feel
> > it's current placement is right, but creating a new facade for a single
> > property looked like an overkill, but now this facade becomes more
> > tangible.
> >
> > 2017-10-25 14:55 GMT+03:00 Vladimir Ozerov <vozerov@gridgain.com>:
> >
> > > Pavel,
> > >
> > > This feature will work independently of streamer. If you want to load
> > data
> > > with streamer, then you disable WAL first through some API call or SQL
> > > command, and then start loading.
> > >
> > > On Wed, Oct 25, 2017 at 2:41 PM, Pavel Tupitsyn <ptupitsyn@apache.org>
> > > wrote:
> > >
> > > > IMO IgniteCache.disableWal() should be enough.
> > > >
> > > > Also what about an option to disable WAL when IgniteDataStreamer is
> > > active?
> > > >
> > > > On Wed, Oct 25, 2017 at 2:38 PM, Vladimir Ozerov <
> vozerov@gridgain.com
> > >
> > > > wrote:
> > > >
> > > > > One more idea - ability to create a cache with initially disabled
> > WAL.
> > > > > Might be useful.
> > > > >
> > > > > On Wed, Oct 25, 2017 at 2:35 PM, Vladimir Ozerov <
> > vozerov@gridgain.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > We have a ticket to implement runtime WAL state management [1].It
> > > will
> > > > be
> > > > > > possible to disable it temporarily. This is very useful for
data
> > > > loading
> > > > > > case. Our experiments show that we can improve data loading
time
> > by a
> > > > > > factor of 2x-10x depending on configuration, indexes and cluster
> > > > > topology.
> > > > > >
> > > > > > We had a private chat with Anton Vinogradov and Alexey Goncharuk,
> > and
> > > > > came
> > > > > > to certain design which you can find inside the ticket. Comments
> > are
> > > > > > welcomed.
> > > > > >
> > > > > > In this email I would like to focus on API of this feature.
> > > > > Considerations:
> > > > > > 1) It should be possible to disable WAL globally or on per-cache
> > > basis.
> > > > > > 2) It should be possible to get current state of WAL for the
> cache.
> > > > > >
> > > > > > Quick draft from my side:
> > > > > > void disableWal();
> > > > > > void disableWal(String... cacheNames);
> > > > > > void enableWal();
> > > > > > void enableWal(String... cacheNames);
> > > > > > boolean isWalEnabled(String... cacheNames);
> > > > > >
> > > > > > Please help improving it.
> > > > > > - Do we really need methods to disable WAL for all caches? This
> is
> > > not
> > > > > > very intuitive feature.
> > > > > > - Where to place these operations? Ignite interface? Ignite
+
> > > > > IgniteCache?
> > > > > > Separate facade?
> > > > > > - Do we need async counterparts?
> > > > > > - Do we need a feature which completes when WAL is enabled back?
> > > > > >
> > > > > > Vladimir.
> > > > > >
> > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-6411
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message