airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Riccomini <criccom...@apache.org>
Subject Re: Airflow 1.8.0 alpha 2
Date Wed, 04 Jan 2017 19:48:54 GMT
Great, I will work toward deploying this in our dev cluster today. :D

On Wed, Jan 4, 2017 at 11:47 AM, Bolke de Bruin <bdbruin@gmail.com> wrote:

> Some issues remain:
>
> * one_failed not executed as dag run is marked failed seemingly
> prematurely (@chris yes you should see this, see below for an example that
> is not working properly), confirmed regression
> * celery instability Alex
> * Wrong DAG state after failure inside branch
>
> So Alpha 2 is definitely not ready for production, but please do put in
> your canary dags and let them run. I am still quite concerned about the
> scheduler integrity and stability.
>
> - Bolke
>
>
>
> one_failed_not_executed.py
> ====
> from airflow import DAG
> from airflow.operators.bash_operator import BashOperator
> from airflow.operators.dummy_operator import DummyOperator
> from datetime import datetime, timedelta
>
> default_args = {
>     'owner': 'airflow',
>     'depends_on_past': False,
>     'start_date': datetime(2016,10,5,19),
>     'email': ['airflow@airflow.com'],
>     'email_on_failure': False,
>     'email_on_retry': False,
>     'retries': 1,
>     'retry_delay': timedelta(seconds=1),
> }
>
> dag = DAG('tutorial', default_args=default_args, schedule_interval='@once')
>
> task1 = BashOperator(
>     task_id='first_one',
>     bash_command='date',
>     dag=dag)
>
> task2 = BashOperator(
>     task_id='second_one',
>     bash_command='this_should_not_work',
>     dag=dag)
>
> task2.set_upstream(task1)
>
>
>
> task3 = BashOperator(
>     task_id='third_one',
>     bash_command='random_command_third',
>     dag=dag)
>
> task3.set_upstream(task2)
>
> fail_task = DummyOperator(
>     task_id='one_failed',
>     trigger_rule='one_failed',
>     dag=dag)
>
> fail_task.set_upstream([task1,task2,task3])
>
> > On 4 Jan 2017, at 20:35, Chris Riccomini <criccomini@apache.org> wrote:
> >
> > Bolke, can you describe the current state of the alpha 2 release? I saw
> > some comments from Alex yesterday about celery instability. If I'm
> running
> > on LocalExecutor, should I be seeing any issues?
> >
> > On Wed, Jan 4, 2017 at 8:20 AM, Bolke de Bruin <bdbruin@gmail.com>
> wrote:
> >
> >> Hi All,
> >>
> >> I have put up Airflow 1.8.0 alpha 2 in https://people.apache.org/~
> bolke/ <
> >> https://people.apache.org/~bolke/> .
> >>
> >> Note: This still cannot be considered an Apache release. Working on
> this.
> >>
> >> This build is signed (note it is served over https).
> >>
> >> Changes are in the area of scheduler stability.
> >>
> >> - Bolke
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message