airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Riccomini <criccom...@apache.org>
Subject Re: Programmatically stop execution of a task
Date Thu, 30 Jun 2016 21:35:16 GMT
@Maxime, is SHUTDOWN needed anymore, then?

On Thu, Jun 30, 2016 at 2:34 PM, Maxime Beauchemin <
maximebeauchemin@gmail.com> wrote:

> There's a new feature in master where tasks instances call the mothership
> (the database) at each heartbeat to see if they are still running. If they
> aren't they essentially commit suicide.
>
> That means that in theory you could have an failure hook that would set the
> running siblings as failed and they should terminate.
>
> Max
>
> On Thu, Jun 30, 2016 at 12:50 PM, Chris Riccomini <criccomini@apache.org>
> wrote:
>
> > Hey Paul,
> >
> > I don't have a solution to this, but I have a suggestion on where you
> might
> > look. There's a state that results in a blue square in the tree view.
> This
> > happens when a running task has its state cleared. It's apparently a
> > 'poison pill' that should cause the task to kill itself. I don't recall
> the
> > name of the state off the top of my head. I *think* it's the SHUTDOWN
> > state. Might trace through that code and see if you can figure it out.
> >
> > Cheers,
> > Chris
> >
> > On Wed, Jun 29, 2016 at 6:02 AM, Ryabchuk, Pavlo <
> > ext-pavlo.ryabchuk@here.com> wrote:
> >
> > > Hi,
> > >
> > > Can someone suggest how can I stop execution/mark(failed or any other
> > > status) of all sibling tasks (also can be passed as a list) if one of
> > them
> > > failed?
> > > It's just useless for me to process other tasks if one failed, because
> in
> > > my case  it's for sure issue with my processing soft and it will lead
> to
> > > fixing and recompilation of it, and in any case reprocessing all tasks
> > with
> > > newer version.
> > > I've tried various manipulations with TaskInstances and Jobs states on
> > > on_failure_callback but nothing seem to work.
> > >
> > > Best,
> > > Paul
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message