airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Richard Baron Penman <richard.pen...@gmail.com>
Subject Re: Terminate task process through UI
Date Wed, 13 Sep 2017 15:37:45 GMT
Thanks Ash.
In my logs I see "Job shut down externally", so that BaseJob.kill()
method is certainly executed.

My understanding is this then calls self.on_kill():
https://github.com/apache/incubator-airflow/blob/b2e1753f5b74ad1b6e0889f7b784ce69623c95ce/airflow/jobs.py#L2512

which terminates the process:
https://github.com/apache/incubator-airflow/blob/b2e1753f5b74ad1b6e0889f7b784ce69623c95ce/airflow/task_runner/bash_task_runner.py#L34

How is the operators on_kill() method then triggered? According to my
logging it is not executed.


On Fri, Sep 8, 2017 at 4:49 AM, Ash Berlin-Taylor
<ash_airflowlist@firemirror.com> wrote:
> That sets the task instance state to SHUTDOWN, which is then picked up by the task sentinel
here https://github.com/apache/incubator-airflow/blob/b2e1753f5b74ad1b6e0889f7b784ce69623c95ce/airflow/jobs.py#L164-L165
<https://github.com/apache/incubator-airflow/blob/b2e1753f5b74ad1b6e0889f7b784ce69623c95ce/airflow/jobs.py#L164-L165>
which triggers kill, so it is meant to work.
>
> -ash
>
>> On 7 Sep 2017, at 20:00, Richard Baron Penman <richard.penman@gmail.com> wrote:
>>
>> Is this known to work?
>> I tried clearing and the on_kill() method wasn't triggered.
>>
>> As far as I can tell the clearing is performed by the
>> clear_task_instances() function, which changes the task status but does not
>> kill the process:
>> https://github.com/apache/incubator-airflow/blob/b2e1753f5b74ad1b6e0889f7b784ce69623c95ce/airflow/models.py#L120
>>
>>
>>
>> On Thu, Sep 7, 2017 at 4:46 AM, Maxime Beauchemin <
>> maximebeauchemin@gmail.com> wrote:
>>
>>> "Clearing" a running task instance either through the UI or through the CLI
>>> should trigger on_kill.
>>>
>>> Max
>>>
>>> On Wed, Sep 6, 2017 at 11:45 AM, Richard Baron Penman <richardbp@gmail.com
>>>>
>>> wrote:
>>>
>>>> Hello,
>>>>
>>>> Is there a way to trigger the on_kill() method of an operator through the
>>>> UI?
>>>> I tried changing the state of a running task and deleting it, but neither
>>>> approach triggered the on_kill() method and so the process was still
>>>> running.
>>>>
>>>> Similar question was asked on SO:
>>>> https://stackoverflow.com/questions/43626844/kill-the-
>>>> airflow-task-running-on-a-remote-location-through-airflow-ui
>>>>
>>>> Richard
>>>>
>>>
>

Mime
View raw message