airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Anand (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AIRFLOW-46) Running Trigger DAG test with SequentialExecutor yields error
Date Wed, 04 May 2016 01:34:12 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-46?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Siddharth Anand updated AIRFLOW-46:
-----------------------------------
    Description: 
On a fresh install of Airflow 1.7.0, executed the following way :  
* git checkout -b b_1.7.0rc1 1.7.0rc1
* python setup.py develop
* airflow initdb (against sqllite and using SequentialExecutor - delete the previous db file)
* airflow webserver -d
* airflow scheduler
* Load examples in the airflow.cfg file and disable all but 2 via the UI as shown below  :
example_trigger_controller_dag and example_trigger_target_dag

The controller dag will run and you will see the following line in the scheduler output to
show it succeeded : 
{panel}
[2016-05-04 02:11:47,680] {__init__.py:36} INFO - Using executor SequentialExecutor
Controller DAG : conditionally_trigger = True
{   'message': 'Hello World'}
[2016-05-04 02:11:51,809] {jobs.py:516} INFO - Prioritizing 0 queued jobs
{panel}

However, when the target dag runs, you will see : 

{noformat}
[2016-05-04 02:11:56,924] {sequential_executor.py:27} INFO - Executing command: airflow run
example_trigger_target_dag bash_task 2016-05-04T01:11:47 --local -sd DAGS_FOLDER/example_dags/example_trigger_target_dag.py

[2016-05-04 02:11:57,870] {__init__.py:36} INFO - Using executor SequentialExecutor
Logging into: /Users/siddharth/Projects/airflow/logs/example_trigger_target_dag/bash_task/2016-05-04T01:11:47
[2016-05-04 02:11:59,008] {__init__.py:36} INFO - Using executor SequentialExecutor
[2016-05-04 02:12:03,131] {sequential_executor.py:27} INFO - Executing command: airflow run
example_trigger_target_dag run_this 2016-05-04T01:11:47 --local -sd DAGS_FOLDER/example_dags/example_trigger_target_dag.py

[2016-05-04 02:12:04,056] {__init__.py:36} INFO - Using executor SequentialExecutor
Logging into: /Users/siddharth/Projects/airflow/logs/example_trigger_target_dag/run_this/2016-05-04T01:11:47
[2016-05-04 02:12:05,179] {__init__.py:36} INFO - Using executor SequentialExecutor
Traceback (most recent call last):
  File "/Users/siddharth/Projects/airflow/venv/bin/airflow", line 6, in <module>
    exec(compile(open(__file__).read(), __file__, 'exec'))
  File "/Users/siddharth/Projects/airflow/airflow/bin/airflow", line 15, in <module>
    args.func(args)
  File "/Users/siddharth/Projects/airflow/airflow/bin/cli.py", line 203, in run
    pool=args.pool,
  File "/Users/siddharth/Projects/airflow/airflow/models.py", line 1067, in run
    result = task_copy.execute(context=context)
  File "/Users/siddharth/Projects/airflow/airflow/operators/python_operator.py", line 65,
in execute
    return_value = self.python_callable(*self.op_args, **self.op_kwargs)
  File "/Users/siddharth/Projects/airflow/airflow/example_dags/example_trigger_target_dag.py",
line 37, in run_this_func
    print( "Remotely received value of {} for key=message".format(kwargs['dag_run'].conf['message']))
AttributeError: 'NoneType' object has no attribute 'conf'
[2016-05-04 02:12:09,318] {jobs.py:516} INFO - Prioritizing 
{noformat}


  was:
On a fresh install of Airflow 1.7.0, executed the following way :  
* git checkout -b b_1.7.0rc1 1.7.0rc1
* python setup.py develop
* airflow initdb (against sqllite and using SequentialExecutor - delete the previous db file)
* airflow webserver -d
* airflow scheduler
* Load examples in the airflow.cfg file and disable all but 2 via the UI as shown below  :
example_trigger_controller_dag and example_trigger_target_dag

The controller dag will run and you will see the following line in the scheduler output to
show it succeeded : 
{panel}
[2016-05-04 02:11:47,680] {__init__.py:36} INFO - Using executor SequentialExecutor
Controller DAG : conditionally_trigger = True
{   'message': 'Hello World'}
[2016-05-04 02:11:51,809] {jobs.py:516} INFO - Prioritizing 0 queued jobs
{panel}

However, when the target dag runs, you will see : 

{panel}
[2016-05-04 02:11:56,924] {sequential_executor.py:27} INFO - Executing command: airflow run
example_trigger_target_dag bash_task 2016-05-04T01:11:47 --local -sd DAGS_FOLDER/example_dags/example_trigger_target_dag.py

[2016-05-04 02:11:57,870] {__init__.py:36} INFO - Using executor SequentialExecutor
Logging into: /Users/siddharth/Projects/airflow/logs/example_trigger_target_dag/bash_task/2016-05-04T01:11:47
[2016-05-04 02:11:59,008] {__init__.py:36} INFO - Using executor SequentialExecutor
[2016-05-04 02:12:03,131] {sequential_executor.py:27} INFO - Executing command: airflow run
example_trigger_target_dag run_this 2016-05-04T01:11:47 --local -sd DAGS_FOLDER/example_dags/example_trigger_target_dag.py

[2016-05-04 02:12:04,056] {__init__.py:36} INFO - Using executor SequentialExecutor
Logging into: /Users/siddharth/Projects/airflow/logs/example_trigger_target_dag/run_this/2016-05-04T01:11:47
[2016-05-04 02:12:05,179] {__init__.py:36} INFO - Using executor SequentialExecutor
Traceback (most recent call last):
  File "/Users/siddharth/Projects/airflow/venv/bin/airflow", line 6, in <module>
    exec(compile(open(__file__).read(), __file__, 'exec'))
  File "/Users/siddharth/Projects/airflow/airflow/bin/airflow", line 15, in <module>
    args.func(args)
  File "/Users/siddharth/Projects/airflow/airflow/bin/cli.py", line 203, in run
    pool=args.pool,
  File "/Users/siddharth/Projects/airflow/airflow/models.py", line 1067, in run
    result = task_copy.execute(context=context)
  File "/Users/siddharth/Projects/airflow/airflow/operators/python_operator.py", line 65,
in execute
    return_value = self.python_callable(*self.op_args, **self.op_kwargs)
  File "/Users/siddharth/Projects/airflow/airflow/example_dags/example_trigger_target_dag.py",
line 37, in run_this_func
    print( "Remotely received value of {} for key=message".format(kwargs['dag_run'].conf['message']))
AttributeError: 'NoneType' object has no attribute 'conf'
[2016-05-04 02:12:09,318] {jobs.py:516} INFO - Prioritizing 
{panel}


> Running Trigger DAG test with SequentialExecutor yields error
> -------------------------------------------------------------
>
>                 Key: AIRFLOW-46
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-46
>             Project: Apache Airflow
>          Issue Type: Bug
>    Affects Versions: Airflow 1.7.0
>         Environment: virtualenv, python 2.7.9
>            Reporter: Siddharth Anand
>            Assignee: Siddharth Anand
>         Attachments: Screenshot 2016-05-04 02.22.28.png
>
>
> On a fresh install of Airflow 1.7.0, executed the following way :  
> * git checkout -b b_1.7.0rc1 1.7.0rc1
> * python setup.py develop
> * airflow initdb (against sqllite and using SequentialExecutor - delete the previous
db file)
> * airflow webserver -d
> * airflow scheduler
> * Load examples in the airflow.cfg file and disable all but 2 via the UI as shown below
 : example_trigger_controller_dag and example_trigger_target_dag
> The controller dag will run and you will see the following line in the scheduler output
to show it succeeded : 
> {panel}
> [2016-05-04 02:11:47,680] {__init__.py:36} INFO - Using executor SequentialExecutor
> Controller DAG : conditionally_trigger = True
> {   'message': 'Hello World'}
> [2016-05-04 02:11:51,809] {jobs.py:516} INFO - Prioritizing 0 queued jobs
> {panel}
> However, when the target dag runs, you will see : 
> {noformat}
> [2016-05-04 02:11:56,924] {sequential_executor.py:27} INFO - Executing command: airflow
run example_trigger_target_dag bash_task 2016-05-04T01:11:47 --local -sd DAGS_FOLDER/example_dags/example_trigger_target_dag.py

> [2016-05-04 02:11:57,870] {__init__.py:36} INFO - Using executor SequentialExecutor
> Logging into: /Users/siddharth/Projects/airflow/logs/example_trigger_target_dag/bash_task/2016-05-04T01:11:47
> [2016-05-04 02:11:59,008] {__init__.py:36} INFO - Using executor SequentialExecutor
> [2016-05-04 02:12:03,131] {sequential_executor.py:27} INFO - Executing command: airflow
run example_trigger_target_dag run_this 2016-05-04T01:11:47 --local -sd DAGS_FOLDER/example_dags/example_trigger_target_dag.py

> [2016-05-04 02:12:04,056] {__init__.py:36} INFO - Using executor SequentialExecutor
> Logging into: /Users/siddharth/Projects/airflow/logs/example_trigger_target_dag/run_this/2016-05-04T01:11:47
> [2016-05-04 02:12:05,179] {__init__.py:36} INFO - Using executor SequentialExecutor
> Traceback (most recent call last):
>   File "/Users/siddharth/Projects/airflow/venv/bin/airflow", line 6, in <module>
>     exec(compile(open(__file__).read(), __file__, 'exec'))
>   File "/Users/siddharth/Projects/airflow/airflow/bin/airflow", line 15, in <module>
>     args.func(args)
>   File "/Users/siddharth/Projects/airflow/airflow/bin/cli.py", line 203, in run
>     pool=args.pool,
>   File "/Users/siddharth/Projects/airflow/airflow/models.py", line 1067, in run
>     result = task_copy.execute(context=context)
>   File "/Users/siddharth/Projects/airflow/airflow/operators/python_operator.py", line
65, in execute
>     return_value = self.python_callable(*self.op_args, **self.op_kwargs)
>   File "/Users/siddharth/Projects/airflow/airflow/example_dags/example_trigger_target_dag.py",
line 37, in run_this_func
>     print( "Remotely received value of {} for key=message".format(kwargs['dag_run'].conf['message']))
> AttributeError: 'NoneType' object has no attribute 'conf'
> [2016-05-04 02:12:09,318] {jobs.py:516} INFO - Prioritizing 
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message