airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ace Haidrey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-1153) params in HiveOperator constructor can't be passed into Hive execution context
Date Fri, 09 Mar 2018 17:41:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-1153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393256#comment-16393256
] 

Ace Haidrey commented on AIRFLOW-1153:
--------------------------------------

Yes I agree it's important to have the script as decoupled as possible from Airflow or any
other system. Your current options are to pass the vars in the user defined macros or to use
a BashOperator and execute your bash command.

If you have a PR you want reviewed, let me know, because we welcome changes.

> params in HiveOperator constructor can't be passed into Hive execution context
> ------------------------------------------------------------------------------
>
>                 Key: AIRFLOW-1153
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1153
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: hive_hooks, operators
>    Affects Versions: Airflow 2.0, Airflow 1.8
>            Reporter: Xianping lin
>            Priority: Critical
>              Labels: easyfix, newbie
>
> params parameter in HiveOperator can't be imported into Hive executation context.
>     so the following centence won't work, because 'mynumber' doesn't work for sql sentence.
>     test_hiveoperator = HiveOperator(
>         task_id='hive_test',
>         hiveconf_jinja_translate=True,
>         hql = ''' use myDB;
>             INSERT OVERWRITE TABLE t2
>             select * from t1 where t1.x > ' ${hiveconf:mynumber}'
>         ''',
>         params={'mynumber': 2},
>         dag=dag
>     )
>     this modification pass the 'params' in HiveOperator construction to Hive sql execution
context.
>     The the variable definition can pass to hive sql



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message