airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vardanguptacse@gmail.com <vardangupta...@gmail.com>
Subject Re: Task is Stuck in Up_For_Retry
Date Tue, 21 Aug 2018 13:39:00 GMT
Is there any possibility that on call of function _check_and_change_state_before_execution
at https://github.com/apache/incubator-airflow/blob/v1-9-stable/airflow/jobs.py#L2500, this
method is not actually being called https://github.com/apache/incubator-airflow/blob/v1-9-stable/airflow/models.py#L1299?
Because even in a happy scenario, no logs is printed from method's implementation and directly
control is reaching here https://github.com/apache/incubator-airflow/blob/v1-9-stable/airflow/jobs.py#L2512
while in stuck phase, we are seeing this log https://github.com/apache/incubator-airflow/blob/v1-9-stable/airflow/jobs.py#L2508
i.e. Task is not able to be run, FYI we've not set any sort of dependency with dag.

Regards,
Vardan Gupta

On 2018/08/16 08:25:37, ramandumcs@gmail.com <ramandumcs@gmail.com> wrote: 
> Hi All,
> 
> We are using airflow 1.9 with Local Executor more. Intermittently we are observing that
tasks are getting stuck in "up_for_retry" mode and are getting retried again and again exceeding
their configured max retries count. like we have configured max retries as 2 but task is retried
15 times and got stuck in up_for_retry state.
> Any pointer on this would be helpful.
> 
> Thanks,
> Raman Gupta
> 

Mime
View raw message