apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amol Kekre <a...@datatorrent.com>
Subject Re: Sequencing of operator calls
Date Sat, 19 Sep 2015 17:19:14 GMT
Bhupesh,
If you want to guaranteed that checkpoint is fully done before next tuple
is processed, you will need to use blocking checkpoint attribute.

Thks
Amol

On Sat, Sep 19, 2015 at 8:59 AM, Sandesh Hegde <sandesh@datatorrent.com>
wrote:

> Blocking checkpoint is supported, please refer to the following jira,
> https://malhar.atlassian.net/browse/APEX-19
>
> On Fri, Sep 18, 2015 at 9:08 PM Bhupesh Chawda <bhupesh@datatorrent.com>
> wrote:
>
> > Thanks for the replies.
> > My intention was to ask if the call to next begin window will start only
> > after the checkpointed call has done it's job.
> > So if just the call is sequenced, but it's a different thread, then it
> does
> > not really help in my case. I need to finish the job which I need to do
> in
> > checkpointed call before any new tuples are processed.
> > On 17-Sep-2015 5:56 am, "Chetan Narsude" <chetan@datatorrent.com> wrote:
> >
> > > Short answer is yes.
> > >
> > > All the control tuples are scheduled to be delivered outside of the
> > window.
> > > As checkpointed callback is triggered because of CHECKPOINT control
> > tuple,
> > > it will happen after endWindow and before the next beginWindow.
> > >
> > > The windowId supplied to CheckpointListener and the one provided to
> > > Operator need not match even though the sequence is defined. So I am
> > > curious how you intend to use this knowledge.
> > >
> > > --
> > > Chetan
> > >
> > >
> > > On Tue, Sep 15, 2015 at 8:31 AM, Thomas Weise <thomas@datatorrent.com>
> > > wrote:
> > >
> > > > It has not changed the operator execution model. State serialization
> is
> > > > still synchronous, write to HDFS is taken out of the operator thread.
> > > >
> > > > On Tue, Sep 15, 2015 at 8:18 AM, Amol Kekre <amol@datatorrent.com>
> > > wrote:
> > > >
> > > > >
> > > > > Sent too soon. Has asynchronous checkpointing changed this?
> > > > >
> > > > > Amol
> > > > >
> > > > > Sent from my iPhone
> > > > >
> > > > > > On Sep 15, 2015, at 12:38 AM, Bhupesh Chawda <
> > > bhupesh@datatorrent.com>
> > > > > wrote:
> > > > > >
> > > > > > Hi All,
> > > > > >
> > > > > > Is it safe to assume that the checkpointed() and the
> beginWindow()
> > > > calls
> > > > > > are sequenced?
> > > > > > In other words, are these calls part of the same thread and
may
> not
> > > run
> > > > > in
> > > > > > parallel?
> > > > > >
> > > > > > Thanks.
> > > > > >
> > > > > > --
> > > > > > -Bhupesh
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message