airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-1734) Sqoop Operator contains logic errors & needs options to pass more sqoop options
Date Sat, 28 Oct 2017 13:09:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-1734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16223494#comment-16223494
] 

ASF subversion and git services commented on AIRFLOW-1734:
----------------------------------------------------------

Commit f6810c9b4f6d821f3fd7ae775606a4295ab8cb20 in incubator-airflow's branch refs/heads/v1-9-test
from [~ahaidrey]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=f6810c9 ]

[AIRFLOW-1734][Airflow 1734] Sqoop hook/operator enhancements

Closes #2703 from Acehaidrey/sqoop_contrib_fixes

(cherry picked from commit 1d531555ecd594ee7ec2c5d3fc87f8d4bcc2c27e)
Signed-off-by: Bolke de Bruin <bolke@xs4all.nl>


> Sqoop Operator contains logic errors & needs options to pass more sqoop options
> -------------------------------------------------------------------------------
>
>                 Key: AIRFLOW-1734
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1734
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: contrib
>            Reporter: Ace Haidrey
>            Assignee: Ace Haidrey
>              Labels: patch
>             Fix For: 1.9.0
>
>
> After taking a look at the Sqoop Operator I have found a number of errors with it. I
will describe them in depth on the PR I will make, but it includes assign wrong parameters,
not hiding the users raw password in the logs (though a method to do that had been created
in the hook, wrong if conditions, and the inability to add other sqoop options that would
make sense to have the option to pass those.
> I have made some remarks in past commits about it too in the case I'm misinterpreting
but regardless there are definitely errors.
> https://github.com/apache/incubator-airflow/pull/2177#pullrequestreview-70046796
> Here is my PR with change by change explanation:
> https://github.com/apache/incubator-airflow/pull/2703/files#diff-8e77f042c2e060bbfd60828431a91e9bL131



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message