apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandesh Hegde <sand...@datatorrent.com>
Subject Re: Sequencing of operator calls
Date Sat, 19 Sep 2015 15:59:38 GMT
Blocking checkpoint is supported, please refer to the following jira,
https://malhar.atlassian.net/browse/APEX-19

On Fri, Sep 18, 2015 at 9:08 PM Bhupesh Chawda <bhupesh@datatorrent.com>
wrote:

> Thanks for the replies.
> My intention was to ask if the call to next begin window will start only
> after the checkpointed call has done it's job.
> So if just the call is sequenced, but it's a different thread, then it does
> not really help in my case. I need to finish the job which I need to do in
> checkpointed call before any new tuples are processed.
> On 17-Sep-2015 5:56 am, "Chetan Narsude" <chetan@datatorrent.com> wrote:
>
> > Short answer is yes.
> >
> > All the control tuples are scheduled to be delivered outside of the
> window.
> > As checkpointed callback is triggered because of CHECKPOINT control
> tuple,
> > it will happen after endWindow and before the next beginWindow.
> >
> > The windowId supplied to CheckpointListener and the one provided to
> > Operator need not match even though the sequence is defined. So I am
> > curious how you intend to use this knowledge.
> >
> > --
> > Chetan
> >
> >
> > On Tue, Sep 15, 2015 at 8:31 AM, Thomas Weise <thomas@datatorrent.com>
> > wrote:
> >
> > > It has not changed the operator execution model. State serialization is
> > > still synchronous, write to HDFS is taken out of the operator thread.
> > >
> > > On Tue, Sep 15, 2015 at 8:18 AM, Amol Kekre <amol@datatorrent.com>
> > wrote:
> > >
> > > >
> > > > Sent too soon. Has asynchronous checkpointing changed this?
> > > >
> > > > Amol
> > > >
> > > > Sent from my iPhone
> > > >
> > > > > On Sep 15, 2015, at 12:38 AM, Bhupesh Chawda <
> > bhupesh@datatorrent.com>
> > > > wrote:
> > > > >
> > > > > Hi All,
> > > > >
> > > > > Is it safe to assume that the checkpointed() and the beginWindow()
> > > calls
> > > > > are sequenced?
> > > > > In other words, are these calls part of the same thread and may not
> > run
> > > > in
> > > > > parallel?
> > > > >
> > > > > Thanks.
> > > > >
> > > > > --
> > > > > -Bhupesh
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message