airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Van Boxel <a...@vanboxel.be>
Subject Re: Airflow 1.8.0 alpha 2
Date Wed, 04 Jan 2017 20:32:29 GMT
I have another fix that certainly need to be in the final release, but not
ready to merge due to failed tests:

https://github.com/apache/incubator-airflow/pull/1961




On Wed, Jan 4, 2017 at 8:48 PM Chris Riccomini <criccomini@apache.org>
wrote:

> Great, I will work toward deploying this in our dev cluster today. :D
>
> On Wed, Jan 4, 2017 at 11:47 AM, Bolke de Bruin <bdbruin@gmail.com> wrote:
>
> > Some issues remain:
> >
> > * one_failed not executed as dag run is marked failed seemingly
> > prematurely (@chris yes you should see this, see below for an example
> that
> > is not working properly), confirmed regression
> > * celery instability Alex
> > * Wrong DAG state after failure inside branch
> >
> > So Alpha 2 is definitely not ready for production, but please do put in
> > your canary dags and let them run. I am still quite concerned about the
> > scheduler integrity and stability.
> >
> > - Bolke
> >
> >
> >
> > one_failed_not_executed.py
> > ====
> > from airflow import DAG
> > from airflow.operators.bash_operator import BashOperator
> > from airflow.operators.dummy_operator import DummyOperator
> > from datetime import datetime, timedelta
> >
> > default_args = {
> >     'owner': 'airflow',
> >     'depends_on_past': False,
> >     'start_date': datetime(2016,10,5,19),
> >     'email': ['airflow@airflow.com'],
> >     'email_on_failure': False,
> >     'email_on_retry': False,
> >     'retries': 1,
> >     'retry_delay': timedelta(seconds=1),
> > }
> >
> > dag = DAG('tutorial', default_args=default_args,
> schedule_interval='@once')
> >
> > task1 = BashOperator(
> >     task_id='first_one',
> >     bash_command='date',
> >     dag=dag)
> >
> > task2 = BashOperator(
> >     task_id='second_one',
> >     bash_command='this_should_not_work',
> >     dag=dag)
> >
> > task2.set_upstream(task1)
> >
> >
> >
> > task3 = BashOperator(
> >     task_id='third_one',
> >     bash_command='random_command_third',
> >     dag=dag)
> >
> > task3.set_upstream(task2)
> >
> > fail_task = DummyOperator(
> >     task_id='one_failed',
> >     trigger_rule='one_failed',
> >     dag=dag)
> >
> > fail_task.set_upstream([task1,task2,task3])
> >
> > > On 4 Jan 2017, at 20:35, Chris Riccomini <criccomini@apache.org>
> wrote:
> > >
> > > Bolke, can you describe the current state of the alpha 2 release? I saw
> > > some comments from Alex yesterday about celery instability. If I'm
> > running
> > > on LocalExecutor, should I be seeing any issues?
> > >
> > > On Wed, Jan 4, 2017 at 8:20 AM, Bolke de Bruin <bdbruin@gmail.com>
> > wrote:
> > >
> > >> Hi All,
> > >>
> > >> I have put up Airflow 1.8.0 alpha 2 in https://people.apache.org/~
> > bolke/ <
> > >> https://people.apache.org/~bolke/> .
> > >>
> > >> Note: This still cannot be considered an Apache release. Working on
> > this.
> > >>
> > >> This build is signed (note it is served over https).
> > >>
> > >> Changes are in the area of scheduler stability.
> > >>
> > >> - Bolke
> >
> >
>
-- 
  _/
_/ Alex Van Boxel

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message