airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From raman gupta <ramandu...@gmail.com>
Subject Re: Task is Stuck in Up_For_Retry
Date Tue, 21 Aug 2018 15:41:14 GMT
One possibility is the unavailability of session while calling
self.task_instance._check_and_change_state_before_execution
function.
(Session is provided via @provide_session decorator)

On Tue, Aug 21, 2018 at 7:09 PM vardanguptacse@gmail.com <
vardanguptacse@gmail.com> wrote:

> Is there any possibility that on call of function
> _check_and_change_state_before_execution at
> https://github.com/apache/incubator-airflow/blob/v1-9-stable/airflow/jobs.py#L2500,
> this method is not actually being called
> https://github.com/apache/incubator-airflow/blob/v1-9-stable/airflow/models.py#L1299?
> Because even in a happy scenario, no logs is printed from method's
> implementation and directly control is reaching here
> https://github.com/apache/incubator-airflow/blob/v1-9-stable/airflow/jobs.py#L2512
> while in stuck phase, we are seeing this log
> https://github.com/apache/incubator-airflow/blob/v1-9-stable/airflow/jobs.py#L2508
> i.e. Task is not able to be run, FYI we've not set any sort of dependency
> with dag.
>
> Regards,
> Vardan Gupta
>
> On 2018/08/16 08:25:37, ramandumcs@gmail.com <ramandumcs@gmail.com>
> wrote:
> > Hi All,
> >
> > We are using airflow 1.9 with Local Executor more. Intermittently we are
> observing that tasks are getting stuck in "up_for_retry" mode and are
> getting retried again and again exceeding their configured max retries
> count. like we have configured max retries as 2 but task is retried 15
> times and got stuck in up_for_retry state.
> > Any pointer on this would be helpful.
> >
> > Thanks,
> > Raman Gupta
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message