Mailing-List: contact commits-help@airflow.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@airflow.incubator.apache.org
Date: Sat, 13 May 2017 10:58:04 +0000 (UTC)
From: "ASF subversion and git services (JIRA)" <jira@apache.org>
To: commits@airflow.incubator.apache.org
Message-ID: <JIRA.13047676.1488461327000.203606.1494673084351@Atlassian.JIRA>
In-Reply-To: <JIRA.13047676.1488461327000@Atlassian.JIRA>
References: <JIRA.13047676.1488461327000@Atlassian.JIRA> <JIRA.13047676.1488461327756@jira-lw-us.apache.org>
Subject: [jira] [Commented] (AIRFLOW-935) Impossible to use plugin executors
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Sat, 13 May 2017 10:58:10 -0000


    [ https://issues.apache.org/jira/browse/AIRFLOW-935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16009265#comment-16009265 ] 

ASF subversion and git services commented on AIRFLOW-935:
---------------------------------------------------------

Commit 08a784ede6ffe1a2389255e1c6d597e9b2131080 in incubator-airflow's branch refs/heads/master from [~stverhae]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=08a784e ]

[AIRFLOW-860][AIRFLOW-935] Fix plugin executor import cycle and executor selection

When a plugin is made with a custom Operator and
executor, an import cycle occurs when the executor
is chosen in airflow.cfg because the
executors/__init__.py starts loading plugins too
early.
changed DEFAULT_EXECUTOR use to a function call
which returns the default executor. This lazy
approach fixes the import cycle.

revision eb5982d (included in 1.8) breaks plugin
executors altogether. It makes a new module for
every plugin, so import statements need to be
adapted, but the executor selection is left
unchanged, so it ends up assigning the plugin
module as an executor.
fixed executor selection to work with the new
plugin modules system introduced in 1.8. in
Airflow.cfg a executor can now be specified as
{plugin_name}.{executor_name}

Fixes:
 -
https://issues.apache.org/jira/browse/AIRFLOW-860
 -
https://issues.apache.org/jira/browse/AIRFLOW-935

Closes #2120 from stverhae/master


> Impossible to use plugin executors
> ----------------------------------
>
>                 Key: AIRFLOW-935
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-935
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: executor, plugins
>    Affects Versions: Airflow 1.8, 1.9.0
>            Reporter: Stijn Verhaegen
>
> revision eb5982d (included in 1.8) breaks plugin executors altogether. It makes a new module for every plugin, so import statements need to be adapted, but the executor selection is left unchanged, so it ends up assigning the plugin module as an executor.
> To reproduce: take any plugin which defines a custom executor and try to get it loaded by setting `executor` in the airflow.cfg


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)