apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timothy Farkas <...@datatorrent.com>
Subject Re: operator recovery window
Date Tue, 15 Dec 2015 07:58:12 GMT
Hi Ashwin,

The recovery checkpoint for operator A is computed by taking the checkpoint
with the largest window id that is less than or equal to the checkpoint
with the largest window id among all the operators down stream to A. The
output operators in a dag will always recover to their most recent
checkpoint. The input operator of the dag may recover to the earliest
checkpoint. Operators between the input and ouput operators could recover
to a window in between.

I don't think you can ever recover to a committed window, the earliest I
think you can recover to is the window after the committed window (may be
wrong on this).

On Mon, Dec 14, 2015 at 11:05 PM, Ashwin Chandra Putta <
ashwinchandrap@gmail.com> wrote:

> In the apex architecture there is concept of checkpointing and concept of
> committed when all operator have crossed a common checkpoint.
>
> So, in which scenarios does a given operator recover at last checkpoint
> window vs last committed window vs some other checkpoint window in between?
> --
>
> Regards,
> Ashwin.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message