airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ján Koščo (JIRA) <j...@apache.org>
Subject [jira] [Created] (AIRFLOW-1050) Retries ignored - regression
Date Tue, 28 Mar 2017 04:17:41 GMT
Ján Koščo created AIRFLOW-1050:
----------------------------------

             Summary: Retries ignored - regression
                 Key: AIRFLOW-1050
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1050
             Project: Apache Airflow
          Issue Type: Bug
    Affects Versions: 1.8.0
            Reporter: Ján Koščo
            Priority: Blocker
         Attachments: Screen Shot 2017-03-28 at 11.15.51.png, Screen Shot 2017-03-28 at 11.15.59.png

SubDag fails when first operator fails, despite the fact it's configured for retries. Information
in UI afterwards are also incorrect. From SubDag prospective it's still {{running}} with operator
marked as {{up_for_retry}}, from main DAG prospective, whole run is marked as {{failed}} same
as SubDag. See attached screenshots. Latest not affected version is RC4 (310fb58). I tested
RC5, 1.8.0 with LocalExecutor and CeleryExecutor.

Example code:
{code}
from datetime import datetime, timedelta

from airflow.models import DAG
from airflow.operators.bash_operator import BashOperator
from airflow.operators.python_operator import PythonOperator
from airflow.operators.subdag_operator import SubDagOperator

args = {
    "start_date": datetime.today(),
}

dag = DAG(
    dag_id="main", default_args=args,
    dagrun_timeout=timedelta(minutes=60),
    schedule_interval=None,
    max_active_runs=1
)

sub_dag = DAG(
    dag_id="main.test",
    default_args=args,
    schedule_interval=None,
)

op = BashOperator(
    task_id="first",
    dag=sub_dag,
    bash_command="echo 1"
)


def throw_error():
    raise RuntimeError()

op2 = PythonOperator(
    task_id="second",
    dag=sub_dag,
    python_callable=throw_error,
    retries=3,
    retry_delay=timedelta(0, 20)
)

op >> op2

prepare_environment = SubDagOperator(
    task_id='test',
    subdag=sub_dag,
    default_args=args,
    dag=dag,
)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message