airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bolke de Bruin <bdbr...@gmail.com>
Subject Re: Airflow (1.6) dag's status "Failed" even when a task was running
Date Tue, 16 Jan 2018 19:08:19 GMT
Airflow 1.6 ??!!

I don’t think any of the maintainers is running that version anymore.

> On 16 Jan 2018, at 19:30, Michael Gong <gonwg@hotmail.com> wrote:
> 
> Hi, Airflow developers,
> 
> 
> On Airflow 1.6, we found sometimes a dag's status was set to "Failed" even when some
tasks were still running.
> 
> 
> The steps to illustrate this behavior is: (suppose a dag contains 2 tasks , A and B.)
> 
> 
>  1.
> let task A and B starts running. --> The dag status is "Running".
>  2.
> Clear task A. --> the task A process will be killed and the task A will set to "Failed".
>  3.
> Task B is still running.
>  4.
> The dag status becomes "Failed", even though the Task B is still running.
> 
> 
> I wonder whether this is an Airflow bug or an expected behavior.
> 
> 
> 
> Any comments are welcome.
> 
> 
> Thanks.
> 
> Michael
> 
> 
> 
> 
> 
> 
> 
> ________________________________
> From: Michael Gong <gonwg@hotmail.com>
> Sent: Friday, March 3, 2017 8:56 PM
> To: dev@airflow.incubator.apache.org
> Subject: Re: Airflow running different with different user id ?
> 
> Hi, Dan,
> 
> 
> Thanks for the encouraging news.
> 
> 
> Maybe you could direct me to the documentation about "run_as" so I could see whether
it is for my purpose.
> 
> 
> I guess it will be passed as an argument for the DAG object , right ?
> 
> 
> Thanks again.
> 
> 
> Michael
> 
> 
> 
> 
> ________________________________
> From: Dan Davydov <dan.davydov@airbnb.com.INVALID>
> Sent: Friday, March 3, 2017 8:44 PM
> To: dev@airflow.incubator.apache.org
> Subject: Re: Airflow running different with different user id ?
> 
> Within a couple of weeks.
> 
> On Fri, Mar 3, 2017 at 12:34 PM, Michael Gong <gonwg@hotmail.com> wrote:
> 
>> When approximately will it be released?
>> 
>> Sent from my PP•KING™ smartphone
>> 
>> On Mar 3, 2017 1:42 PM, Dan Davydov <dan.davydov@airbnb.com.INVALID>
>> wrote:
>> Yes it is starting on 1.8.0 which will be released soon, you can look in
>> the documentation/grep for "run_as".
>> 
>> On Mar 3, 2017 8:50 AM, "Michael Gong" <gonwg@hotmail.com> wrote:
>> 
>>> Hi,
>>> 
>>> 
>>> Suppose I have 1 airflow instance running 2 different DAGs, is it
>> possible
>>> to specify the 2 DAGs running under 2 different ids ?
>>> 
>>> 
>>> Any advises are welcomed.
>>> 
>>> 
>>> Thanks.
>>> 
>>> Michael
>>> 
>>> 
>>> 
>>> 
>>> 
>> 


Mime
View raw message