airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Beau Barker (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AIRFLOW-3102) Processor exited with return code -9
Date Tue, 25 Sep 2018 03:49:00 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-3102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Beau Barker updated AIRFLOW-3102:
---------------------------------
    Description: 
I am finding tasks aren't starting for me.

The scheduler logs are showing "Processor exited with return code -9".
{code:java}
airflow.utils.dag_processing.DagFileProcessorManager INFO: Started a process (PID: 23) to
generate tasks for /usr/local/airflow/dags/definitions/my_dag.py
airflow.jobs.SchedulerJob INFO:
================================================================================
DAG File Processing Stats
File Path PID Runtime Last Runtime Last Run
----------------------------------------------------------- ----- --------- --------------
-------------------
/usr/local/airflow/dags/definitions/my_dag.py 23 2.39s 7.66s 2018-09-22T01:27:53
================================================================================
airflow.utils.dag_processing.DagFileProcessorManager INFO: Processor for /usr/local/airflow/dags/definitions/my_dag.py
finished
airflow.utils.dag_processing.DagFileProcessorManager WARNING: Processor for /usr/local/airflow/dags/definitions/my_dag.py
exited with return code -9.
{code}
 After much searching I've narrowed it down to a line in logging_mixin.py:
{code:java}
self.logger.log(self.level, self._buffer)
{code}
Link on Github:
 [https://github.com/apache/incubator-airflow/blob/master/airflow/utils/log/logging_mixin.py#L95]

Commenting that line, the DagFileProcessorManager succeeds.

 

  was:
I am finding tasks aren't starting for me.

The scheduler logs are showing "Processor exited with return code -9".
{code:java}
airflow.utils.dag_processing.DagFileProcessorManager INFO: Started a process (PID: 23) to
generate tasks for /usr/local/airflow/dags/definitions/my_dag.py
airflow.jobs.SchedulerJob INFO:
================================================================================
DAG File Processing Stats
File Path PID Runtime Last Runtime Last Run
----------------------------------------------------------- ----- --------- --------------
-------------------
/usr/local/airflow/dags/definitions/my_dag.py 23 2.39s 7.66s 2018-09-22T01:27:53
================================================================================
airflow.utils.dag_processing.DagFileProcessorManager INFO: Processor for /usr/local/airflow/dags/definitions/my_dag.py
finished
airflow.utils.dag_processing.DagFileProcessorManager WARNING: Processor for /usr/local/airflow/dags/definitions/my_dag.py
exited with return code -9.
{code}
 After much searching I've narrowed it down to a line in logging_mixin.py:
{code:java}
self.logger.log(self.level, self._buffer)
{code}
Link on Github:
 [https://github.com/apache/incubator-airflow/blob/master/airflow/utils/log/logging_mixin.py#L95]

Commenting that line, the DagFileProcessorManager succeeds.

 

*Solution*

This ended up being due to the airflow.processor logger being set to "propagate".

Closing!


> Processor exited with return code -9
> ------------------------------------
>
>                 Key: AIRFLOW-3102
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3102
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 1.10.0
>         Environment: Docker
>            Reporter: Beau Barker
>            Assignee: Beau Barker
>            Priority: Major
>
> I am finding tasks aren't starting for me.
> The scheduler logs are showing "Processor exited with return code -9".
> {code:java}
> airflow.utils.dag_processing.DagFileProcessorManager INFO: Started a process (PID: 23)
to generate tasks for /usr/local/airflow/dags/definitions/my_dag.py
> airflow.jobs.SchedulerJob INFO:
> ================================================================================
> DAG File Processing Stats
> File Path PID Runtime Last Runtime Last Run
> ----------------------------------------------------------- ----- --------- --------------
-------------------
> /usr/local/airflow/dags/definitions/my_dag.py 23 2.39s 7.66s 2018-09-22T01:27:53
> ================================================================================
> airflow.utils.dag_processing.DagFileProcessorManager INFO: Processor for /usr/local/airflow/dags/definitions/my_dag.py
finished
> airflow.utils.dag_processing.DagFileProcessorManager WARNING: Processor for /usr/local/airflow/dags/definitions/my_dag.py
exited with return code -9.
> {code}
>  After much searching I've narrowed it down to a line in logging_mixin.py:
> {code:java}
> self.logger.log(self.level, self._buffer)
> {code}
> Link on Github:
>  [https://github.com/apache/incubator-airflow/blob/master/airflow/utils/log/logging_mixin.py#L95]
> Commenting that line, the DagFileProcessorManager succeeds.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message