ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Ozerov <voze...@gridgain.com>
Subject Re: Ability to disable WAL in runtime to speed up load
Date Wed, 25 Oct 2017 14:20:53 GMT
Alex,

My bad, I meant "isWalEnabled(String cacheName)" of course.

On Wed, Oct 25, 2017 at 3:34 PM, Anton Vinogradov <avinogradov@gridgain.com>
wrote:

> Pavel,
>
> WAL disabling is a very dangerous operation and it seems to be not a good
> idea to allow run regular operation with .disabledWal().
> Let's think twice how to make new API safe.
>
> On Wed, Oct 25, 2017 at 3:25 PM, Pavel Tupitsyn <ptupitsyn@apache.org>
> wrote:
>
> > Vladimir,
> >
> > It would be useful to be able to automatically disable WAL when streaming
> > starts
> > and re-enable after it ends, don't you think so?
> >
> > Something like IgniteDataStreamer.disableWal property.
> >
> > This is in addition to other API calls that you suggested.
> >
> > On Wed, Oct 25, 2017 at 3:25 PM, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com> wrote:
> >
> > > I do not like boolean isWalEnabled(String... cacheNames) - it's
> semantics
> > > is confusing. Should it return true if WAL is enabled for all caches or
> > if
> > > WAL is enabled for at least one cache? IMO, since this is a local-read
> > > operation, single cache per call is enough.
> > >
> > > As for the API placement, it looks like we need another facade
> > > (IgniteControl ?) where active(bool) should also be moved. I do not
> feel
> > > it's current placement is right, but creating a new facade for a single
> > > property looked like an overkill, but now this facade becomes more
> > > tangible.
> > >
> > > 2017-10-25 14:55 GMT+03:00 Vladimir Ozerov <vozerov@gridgain.com>:
> > >
> > > > Pavel,
> > > >
> > > > This feature will work independently of streamer. If you want to load
> > > data
> > > > with streamer, then you disable WAL first through some API call or
> SQL
> > > > command, and then start loading.
> > > >
> > > > On Wed, Oct 25, 2017 at 2:41 PM, Pavel Tupitsyn <
> ptupitsyn@apache.org>
> > > > wrote:
> > > >
> > > > > IMO IgniteCache.disableWal() should be enough.
> > > > >
> > > > > Also what about an option to disable WAL when IgniteDataStreamer
is
> > > > active?
> > > > >
> > > > > On Wed, Oct 25, 2017 at 2:38 PM, Vladimir Ozerov <
> > vozerov@gridgain.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > One more idea - ability to create a cache with initially disabled
> > > WAL.
> > > > > > Might be useful.
> > > > > >
> > > > > > On Wed, Oct 25, 2017 at 2:35 PM, Vladimir Ozerov <
> > > vozerov@gridgain.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > We have a ticket to implement runtime WAL state management
> [1].It
> > > > will
> > > > > be
> > > > > > > possible to disable it temporarily. This is very useful
for
> data
> > > > > loading
> > > > > > > case. Our experiments show that we can improve data loading
> time
> > > by a
> > > > > > > factor of 2x-10x depending on configuration, indexes and
> cluster
> > > > > > topology.
> > > > > > >
> > > > > > > We had a private chat with Anton Vinogradov and Alexey
> Goncharuk,
> > > and
> > > > > > came
> > > > > > > to certain design which you can find inside the ticket.
> Comments
> > > are
> > > > > > > welcomed.
> > > > > > >
> > > > > > > In this email I would like to focus on API of this feature.
> > > > > > Considerations:
> > > > > > > 1) It should be possible to disable WAL globally or on
> per-cache
> > > > basis.
> > > > > > > 2) It should be possible to get current state of WAL for
the
> > cache.
> > > > > > >
> > > > > > > Quick draft from my side:
> > > > > > > void disableWal();
> > > > > > > void disableWal(String... cacheNames);
> > > > > > > void enableWal();
> > > > > > > void enableWal(String... cacheNames);
> > > > > > > boolean isWalEnabled(String... cacheNames);
> > > > > > >
> > > > > > > Please help improving it.
> > > > > > > - Do we really need methods to disable WAL for all caches?
This
> > is
> > > > not
> > > > > > > very intuitive feature.
> > > > > > > - Where to place these operations? Ignite interface? Ignite
+
> > > > > > IgniteCache?
> > > > > > > Separate facade?
> > > > > > > - Do we need async counterparts?
> > > > > > > - Do we need a feature which completes when WAL is enabled
> back?
> > > > > > >
> > > > > > > Vladimir.
> > > > > > >
> > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-6411
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message