airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxime Beauchemin <maximebeauche...@gmail.com>
Subject Re: execution_timeout ignored for PythonOperator in 1.7.1.2?
Date Wed, 08 Jun 2016 15:16:18 GMT
Are you using the LocalExecutor? there's a limitation around using the task
timeout in LocalExecutor under some circumstances. If that's the case you
should see this somewhere in your logs:
https://github.com/apache/incubator-airflow/blob/master/airflow/utils/timeout.py#L45

We may want to move the logic enforcing the timeout from the context
manager/signal approach to BaseJob.

Max

On Tue, Jun 7, 2016 at 5:47 PM, Alyssa Kwan <alyssa.kwan@cloverhealth.com>
wrote:

> Hi,
>
> We're having trouble with a `PythonOperator`'s `execution_timeout`. It's
> being set directly in the constructor as `datetime.timedelta(minutes=10)`.
> However, the task doesn't timeout until almost 5 hours is passed. This
> behavior is pretty consistent. It's merely a SQL query issued by the Python
> callable that can get deadlocked.
>
> Any thoughts?
>
> Thanks,
> Alyssa

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message