airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-1150) Fix scripts execution in SparkSql hook
Date Fri, 12 May 2017 09:40:04 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-1150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16007872#comment-16007872
] 

ASF subversion and git services commented on AIRFLOW-1150:
----------------------------------------------------------

Commit f97bc5ed3dc42f975776175f7a269c0604f49123 in incubator-airflow's branch refs/heads/master
from [~lanzani]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=f97bc5e ]

[AIRFLOW-1150] Fix scripts execution in sparksql hook[]

When using the the SparkSqlOperator and submitting
a file (ending with
`.sql` or `.hql`), a whitespace need to be
appended, otherwise a Jinja
error will be raised.

However the trailing whitespace confused the hook
as those files will
not end with `.sql` and `.hql`, but with `.sql `
and `.hql `. This PR
fixes this.

In the test, I've added the `get_after` function
to easily check if the
path is really stripped or not by the `-f` option.

Closes #2259 from gglanzani/master


> Fix scripts execution in SparkSql hook 
> ---------------------------------------
>
>                 Key: AIRFLOW-1150
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1150
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: contrib
>    Affects Versions: Airflow 1.8
>            Reporter: Giovanni Lanzani
>             Fix For: 1.8.2
>
>
> When using the the SparkSqlOperator and submitting a file (ending with .sql or .hql),
a whitespace need to be appended, otherwise a Jinja error will be raised. However the trailing
whitespace confused the hook as those files will not end with .sql and .hql, but with .sql
and .hql. This PR fixes this.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message