airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxime Beauchemin <maximebeauche...@gmail.com>
Subject Re: execution_timeout ignored for PythonOperator in 1.7.1.2?
Date Wed, 08 Jun 2016 15:22:38 GMT
I created this JIRA to document the issue.
https://issues.apache.org/jira/browse/AIRFLOW-221

Max

On Wed, Jun 8, 2016 at 8:16 AM, Maxime Beauchemin <
maximebeauchemin@gmail.com> wrote:

> Are you using the LocalExecutor? there's a limitation around using the
> task timeout in LocalExecutor under some circumstances. If that's the case
> you should see this somewhere in your logs:
>
> https://github.com/apache/incubator-airflow/blob/master/airflow/utils/timeout.py#L45
>
> We may want to move the logic enforcing the timeout from the context
> manager/signal approach to BaseJob.
>
> Max
>
> On Tue, Jun 7, 2016 at 5:47 PM, Alyssa Kwan <alyssa.kwan@cloverhealth.com>
> wrote:
>
>> Hi,
>>
>> We're having trouble with a `PythonOperator`'s `execution_timeout`. It's
>> being set directly in the constructor as `datetime.timedelta(minutes=10)`.
>> However, the task doesn't timeout until almost 5 hours is passed. This
>> behavior is pretty consistent. It's merely a SQL query issued by the Python
>> callable that can get deadlocked.
>>
>> Any thoughts?
>>
>> Thanks,
>> Alyssa
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message