airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-2773) DataFlowPythonOperator does not handle correctly task_id containing underscores
Date Mon, 23 Jul 2018 11:10:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-2773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552668#comment-16552668
] 

ASF subversion and git services commented on AIRFLOW-2773:
----------------------------------------------------------

Commit 1656209cbe31da9f4bd10b6eead8d9f65198c04b in incubator-airflow's branch refs/heads/master
from [~kaxilnaik]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=1656209 ]

[AIRFLOW-2773] Validates Dataflow Job Name

Closes #3623 from kaxil/AIRFLOW-2773


> DataFlowPythonOperator does not handle correctly task_id containing underscores
> -------------------------------------------------------------------------------
>
>                 Key: AIRFLOW-2773
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2773
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: Dataflow
>    Affects Versions: 1.9.0
>            Reporter: Evgeny Podlepaev
>            Priority: Minor
>
> DataFlowPythonOperator generates a job name that does not get accepted by Dataflow API
when task_id contains underscores. Example: DataFlowPythonOperator(task_id='analyze_search_results',
...)
> will lead to the following error:
> ValueError: Pipeline has validations errors: Invalid job_name (analyze_search_results-02e17268);
the name must consist of only the characters [-a-z0-9], starting with a letter and ending
with a letter or number.
> The fix seems to be as simple as changing DataFlowHook.start_python_dataflow() to do
> name = task_id.replace('_', '-')



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message