Mailing-List: contact dev-help@apex.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@apex.apache.org
MIME-Version: 1.0
In-Reply-To: <CAHvM9d2A8_AzmA7Bfqw2hc6vrOnDo8jnB7uSktNFxg7kUNpghg@mail.gmail.com>
References: <CAHvM9d0_mmH53+dgiQ2b5smRGsiPVdfayK0iEJKChb+b4_BxKw@mail.gmail.com>
 <CAD0KA=PTJ_3b94Mw2Wfu=u77woo=O5Hzfokp8GjdAbhw8P1uag@mail.gmail.com>
 <CAHvM9d3zjOWOn3Qors2CWRaffKwG4PjHFqMNvLyYCm7-8+Leug@mail.gmail.com>
 <CA+mX7frkFJb1t0A0=fh9nyETCdjc5OXaA1Y2f3YDWuPPOJCkow@mail.gmail.com>
 <CAHvM9d3tDxfmNd2D3h_aWj8Yn0wbG=j2CfA-riiTpYSaUX96zQ@mail.gmail.com>
 <CAMaZnT59YnpqN9QrAkx+AHHKSckTuh67=R0qz5L94zahj+7LEQ@mail.gmail.com> <CAHvM9d2A8_AzmA7Bfqw2hc6vrOnDo8jnB7uSktNFxg7kUNpghg@mail.gmail.com>
From: Amol Kekre <amol@datatorrent.com>
Date: Wed, 18 Jan 2017 07:31:11 -0800
Message-ID: <CAMaZnT51cAbMgSF_7uDD71WO0=+By9toas-2q4GsHdhiZmrJWw@mail.gmail.com>
Subject: Re: Shutdown of an Apex app
To: dev@apex.apache.org
Content-Type: multipart/alternative; boundary=001a1140e70a4757330546601b0a
archived-at: Wed, 18 Jan 2017 15:31:21 -0000

--001a1140e70a4757330546601b0a
Content-Type: text/plain; charset=UTF-8

Agreed on input adapter way.

Enforcing all input adapters send shutdown on the same window cannot be
guaranteed, so there will be a corner case where windows are not aligned.
An ask from Stram to send shutdown way in the future reduces the scope of
the corner case, but then defeats the purpose. Even for a single logical
input adapter, all its partitions will need to issue shutdown control tuple.

So a best case attempt to align within at a window should be done.

Thks
Amol


On Wed, Jan 18, 2017 at 12:14 AM, Bhupesh Chawda <bhupesh@datatorrent.com>
wrote:

> Yes, Amol. That will be the case for a kill type of shutdown.
> But for graceful shutdown, it is important that each operator in the DAG
> processes exactly the same amount of data. That is the reason the shutdown
> needs to start at input operators and stop their functioning as the first
> step. Thereafter the control tuple can propagate down the DAG and shutdown
> operators as and when they encounter the control tuple.
>
> ~ Bhupesh
>
> On Wed, Jan 18, 2017 at 1:10 PM, Amol Kekre <amol@datatorrent.com> wrote:
>
> > Can be done by sending "shutdown" message via heartbeat to Stram. Then on
> > stram can shutdown the entire app
> >
> > Thks
> > Amol
> >
> >
> > On Tue, Jan 17, 2017 at 11:05 PM, Bhupesh Chawda <
> bhupesh@datatorrent.com>
> > wrote:
> >
> > > Yes Ajay, for a graceful shutdown, the data sent out should be
> processed.
> > >
> > > On Wed, Jan 18, 2017 at 12:19 PM, AJAY GUPTA <ajaygit158@gmail.com>
> > wrote:
> > >
> > > > +1 to idea.
> > > >
> > > > Will this ensure downstream operators to process all data received
> > before
> > > > shutdown is called?
> > > > Also, how do we plan to handle cases where 2 sub-DAGs merge to a
> single
> > > > operator somewhere downstream, and an operator in one of the sub-DAGs
> > > sends
> > > > ShutdownException.
> > > >
> > > >
> > > > Ajay
> > > >
> > > > On Wed, Jan 18, 2017 at 12:00 PM, Bhupesh Chawda <
> > > bhupesh@datatorrent.com>
> > > > wrote:
> > > >
> > > > > This JIRA is to stop the DAG in a crude manner, based on an error
> > > > > condition. I think this might also need similar functionality as an
> > > error
> > > > > condition can occur anywhere in the DAG.
> > > > >
> > > > > Perhaps we can modify the same JIRA to include a graceful +
> > ungraceful
> > > > > (kill) shutdown from any operator in the DAG.
> > > > >
> > > > > ~ Bhupesh
> > > > >
> > > > > On Wed, Jan 18, 2017 at 11:55 AM, Tushar Gosavi <
> > > tushar@datatorrent.com>
> > > > > wrote:
> > > > >
> > > > > > I think this would be a great addition for batch use cases or use
> > > > > > cases were DAG needs to be shutdown after detecting some
> > > > > > completion/error condition through the operator. We have one Jira
> > > > > > Opened for such functionality
> > > > > > https://issues.apache.org/jira/browse/APEXCORE-503.
> > > > > >
> > > > > > - Tushar.
> > > > > >
> > > > > >
> > > > > > On Wed, Jan 18, 2017 at 11:45 AM, Bhupesh Chawda
> > > > > > <bhupesh@datatorrent.com> wrote:
> > > > > > > Hi All,
> > > > > > >
> > > > > > > Currently we can shutdown an Apex app in the following ways:
> > > > > > > 1. Throw ShutdownException() from *all* the input operators
> > > > > > > 2. Use Apex CLI to shutdown an app using the YARN App Id
> > > > > > >
> > > > > > > I think we should have some way of shutting down an application
> > > from
> > > > > > within
> > > > > > > an operator. It is not always true that the trigger for
> shutdown
> > is
> > > > > sent
> > > > > > by
> > > > > > > the input operator only. Sometimes, an end condition may be
> > > detected
> > > > by
> > > > > > > some operator in the DAG which wants the processing to end.
> Such
> > a
> > > > > > > shutdown, although triggered from some intermediate operator in
> > the
> > > > > DAG,
> > > > > > > should guarantee graceful shut down of the application.
> > > > > > >
> > > > > > > Thoughts?
> > > > > > >
> > > > > > > ~ Bhupesh
> > > > > >
> > > > >
> > > >
> > >
> >
>

--001a1140e70a4757330546601b0a--