airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vianney FOUCAULT (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AIRFLOW-1184) Contrib Spark Submit Hook does not split argument and argument value
Date Tue, 09 May 2017 16:05:04 GMT
Vianney FOUCAULT created AIRFLOW-1184:
-----------------------------------------

             Summary: Contrib Spark Submit Hook does not split argument and argument value
                 Key: AIRFLOW-1184
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1184
             Project: Apache Airflow
          Issue Type: Bug
          Components: contrib, hooks
    Affects Versions: Airflow 2.0, Airflow 1.8
            Reporter: Vianney FOUCAULT
            Assignee: Vianney FOUCAULT
             Fix For: Airflow 2.0, Airflow 1.8


Python Popen expect a list as command. Spark submit too, as: 
* ['--option value'] 
is not the same as 
* ['--option', 'value']

in regards of spark. eg spark logs : (yarn logs)

Error: Unknown option --end 2017-05-08
Error: Unknown option --begin 2017-05-07
Error: Unknown option --db_name mydb
Error: Missing option --begin
Error: Missing option --end
Error: Missing option --db_name




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message