ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Atri Sharma <atri.j...@gmail.com>
Subject Re: Write-behind store behavior on node stop.
Date Tue, 07 Jul 2015 19:29:51 GMT
Folks,

I thought more about this and came up with a WAL page formal that we can
maintain in a separate queue and flush to persistent store. This allows to
recover any lost queue data and also allows to rebuild a cluster from
scratch.

Dmitriy, I understand that you were inclined to have redundancy of write
behind store buffers for primary node on backup nodes as well and flush
them in case of failure. Does the WAL page idea fit in your mind? Else we
could go with pure redundancy based solution.

Thoughts?
On 6 Jul 2015 16:15, "Atri Sharma" <atri.jiit@gmail.com> wrote:

> Dmitriy,
>
> I created the ticket. Please let me know your thoughts around this.
>
> Once we have something in place, I will create page on wiki.
>
> On Mon, Jul 6, 2015 at 1:47 PM, Dmitriy Setrakyan <dsetrakyan@apache.org>
> wrote:
>
>> On Mon, Jul 6, 2015 at 1:13 AM, Atri Sharma <atri.jiit@gmail.com> wrote:
>>
>> > Sounds good.
>> >
>> > I also propose controlling this behaviour through user settable option
>> > (since this might potentially have some performance implications). So if
>> > the user is doing ad hoc analytics, he/she might not care much about
>> bit of
>> > data loss and might prefer the performance gain instead.
>> >
>>
>> I agree. Atri, can you please create a ticket describing this behavior? We
>> can move this discussion there.
>>
>>
>> >
>> > On Mon, Jul 6, 2015 at 1:40 PM, Dmitriy Setrakyan <
>> dsetrakyan@apache.org>
>> > wrote:
>> >
>> > > On Mon, Jul 6, 2015 at 1:00 AM, Atri Sharma <atri.jiit@gmail.com>
>> wrote:
>> > >
>> > > > Can some sort of persistent Write Ahead Logging help here?
>> > > >
>> > >
>> > > I think so. Perhaps we can have the same type of queue on the backup
>> > nodes
>> > > which will only get flushed in case of primary node failure?
>> > >
>> > >
>> > > >
>> > > > On Mon, Jul 6, 2015 at 11:32 AM, Dmitriy Setrakyan <
>> > > dsetrakyan@apache.org>
>> > > > wrote:
>> > > >
>> > > > > On Sun, Jul 5, 2015 at 10:58 PM, Vladimir Ozerov <
>> > vozerov@gridgain.com
>> > > >
>> > > > > wrote:
>> > > > >
>> > > > > > Igniters,
>> > > > > >
>> > > > > > Can someone explain me how write-behind store should behave
in
>> case
>> > > of
>> > > > > node
>> > > > > > stop when some changes has not been flushed to the underlying
>> store
>> > > > yet?
>> > > > > > Are we guarantee that all pending changes will be flushed
to the
>> > > > > underlying
>> > > > > > store?
>> > > > > >
>> > > > > >
>> > > > > I think we should make the best effort to persist the un-flushed
>> > > updates
>> > > > in
>> > > > > case of graceful exit. Of course, not guarantees can be provided,
>> and
>> > > if
>> > > > a
>> > > > > server stop abruptly, then some updates might be lost.
>> > > > >
>> > > > >
>> > > > > > Vladimir.
>> > > > > >
>> > > > >
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > Regards,
>> > > >
>> > > > Atri
>> > > > *l'apprenant*
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > Regards,
>> >
>> > Atri
>> > *l'apprenant*
>> >
>>
>
>
>
> --
> Regards,
>
> Atri
> *l'apprenant*
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message