airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "EKC (Erik Cederstrand)" <...@novozymes.com>
Subject Re: Fix on_kill command for operators
Date Mon, 15 Jan 2018 08:40:26 GMT
I'm not sure if it's related, but there's an additional issue with attempting to kill processes,
that doesn't always kill the process: https://issues.apache.org/jira/browse/AIRFLOW-949


Kind regards,

Erik Cederstrand

________________________________
From: Milan van der Meer <milan.vandermeer@realimpactanalytics.com>
Sent: Friday, January 12, 2018 9:23:24 PM
To: dev@airflow.incubator.apache.org
Subject: Re: Fix on_kill command for operators

Currently the on_kill does not get triggered when you clear from the UI.
As you mention, adding the 'fix' mentioned in the first comment of the
issue, does no fix the problem as it does not trigger on the right
operators context.

Im not sure what exact changes are planned for the next 1.10 release, but
if the whole UI change is planned, this could be a good opportunity to also
fix this bug.

On Mon, Jan 8, 2018 at 2:55 PM, Ash Berlin-Taylor <
ash_airflowlist@firemirror.com> wrote:

> Without this change does on_kill ever get triggered? It seems like this
> change is desired behaviour.
>
> As per the first comment https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2F&data=01%7C01%7CEKC%40novozymes.com%7C4d2e0189d7e44025c48608d559fa5913%7C43d5f49ee03a4d22a2285684196bb001%7C0&sdata=9%2BWOuDmUe9M1WK2%2FqSsOtsaNrZnTtbvX%2Fq20dOBXVTA%3D&reserved=0
> jira/browse/AIRFLOW-1623?focusedCommentId=16171819&
> page=com.atlassian.jira.plugin.system.issuetabpanels:
> comment-tabpanel#comment-16171819 <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2F&data=01%7C01%7CEKC%40novozymes.com%7C4d2e0189d7e44025c48608d559fa5913%7C43d5f49ee03a4d22a2285684196bb001%7C0&sdata=9%2BWOuDmUe9M1WK2%2FqSsOtsaNrZnTtbvX%2Fq20dOBXVTA%3D&reserved=0
> jira/browse/AIRFLOW-1623?focusedCommentId=16171819&
> page=com.atlassian.jira.plugin.system.issuetabpanels:
> comment-tabpanel#comment-16171819> I'm not sure this is the right fix. It
> also seems like this would end up running the on_kill in a different
> process to the rest of the operator.
>
> I wonder if somewhere a signal handler is missing somewhere in one of the
> `run --local` or `run --raw`. I tried to follow all the paths through from
> ui to sig handlers but got stuck in a tiwsty maze of classes. (and was
> attempting to do it just from reading the code)?
>
>
> > On 8 Jan 2018, at 13:15, Driesprong, Fokko <fokko@driesprong.frl> wrote:
> >
> > Yes, for Spark this should work. Depending on the operator and the
> > implementation:
> > https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fincubator-airflow%2Fblob%2F&data=01%7C01%7CEKC%40novozymes.com%7C4d2e0189d7e44025c48608d559fa5913%7C43d5f49ee03a4d22a2285684196bb001%7C0&sdata=z1p%2FFlZ2cgPo%2BVK9sQJoDDJbKBEIdslsxtGOP%2F%2B7g0M%3D&reserved=0
> 3e6babe8ed8f8f281b67aa3f4e03bf3cfc1bcbaa/airflow/contrib/
> hooks/spark_submit_hook.py#L412-L428
> >
> > However this is a big change in behaviour. I'm curious about the opinion
> of
> > others.
> >
> > Cheers,
> > Fokko
> >
> >
> > 2018-01-08 14:12 GMT+01:00 Milan van der Meer <
> > milan.vandermeer@realimpactanalytics.com>:
> >
> >> Any help? :)
> >>
> >> On Thu, Dec 14, 2017 at 8:12 PM, Milan van der Meer <
> >> milan.vandermeer@realimpactanalytics.com> wrote:
> >>
> >>> I recently openend the following PR: https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2F&data=01%7C01%7CEKC%40novozymes.com%7C4d2e0189d7e44025c48608d559fa5913%7C43d5f49ee03a4d22a2285684196bb001%7C0&sdata=7AHCTsBnf%2F9nW0IY4B5ef3zfeY%2FlrH86WXwCt9YPwiU%3D&reserved=0
> >>> incubator-airflow/pull/2877
> >>>
> >>> The problem is that on_kill is not called for operators when you clear
> a
> >>> task from the UI.
> >>> Thats problematic when working with ex. spark clusters as the jobs on
> the
> >>> cluster need to be killed.
> >>>
> >>> The issue is in the core code of Airflow and Im not familiar enough
> with
> >>> the inner workings there. So I could use some directions on this one
> from
> >>> people who are familiar.
> >>>
> >>> For more info, check out the PR.
> >>>
> >>> Kind regards,
> >>> Milan
> >>>
> >>
> >>
> >>
> >> --
> >>
> >> *Milan van der Meer*
> >>
> >> *Real**Impact* Analytics *| *Big Data Consultant
> >> https://emea01.safelinks.protection.outlook.com/?url=www.realimpactanalytics.com&data=01%7C01%7CEKC%40novozymes.com%7C4d2e0189d7e44025c48608d559fa5913%7C43d5f49ee03a4d22a2285684196bb001%7C0&sdata=863cz5%2FF3LXZh52xM3pD0ORkew1PI8Q1Bt%2B2pRiayWI%3D&reserved=0
> >>
> >> *BE *+32 498 45 96 22 <0032498459622>* | Skype *milan.vandermeer.ria
> >>
>
>


--

*Milan van der Meer*

*Real**Impact* Analytics *| *Big Data Consultant
https://emea01.safelinks.protection.outlook.com/?url=www.realimpactanalytics.com&data=01%7C01%7CEKC%40novozymes.com%7C4d2e0189d7e44025c48608d559fa5913%7C43d5f49ee03a4d22a2285684196bb001%7C0&sdata=863cz5%2FF3LXZh52xM3pD0ORkew1PI8Q1Bt%2B2pRiayWI%3D&reserved=0

*BE *+32 498 45 96 22 <0032498459622>* | Skype *milan.vandermeer.ria

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message