hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vijay Singh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-13817) Allow DNS CNAME ALIAS Resolution from apache hive beeline JDBC URL to allow for failover
Date Mon, 23 May 2016 05:45:12 GMT

    [ https://issues.apache.org/jira/browse/HIVE-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295939#comment-15295939
] 

Vijay Singh commented on HIVE-13817:
------------------------------------

Also verified that hive2 action invoked from oozie workflow with CNAME alias string and useCanonicalName=true
successfully obtains delegationtokens and executes the workflow. Please see invocation and
verfication log excerpts.
{code}
[root@vjd-1 hive2]# cat hive2job.properties 
nameNode=hdfs://vjd-1.gce.cloudera.com:8020
jobTracker=yarnRM
queueName=default
jdbcURL=jdbc:hive2://vjlb.gce.cloudera.com:10000/default;useCanonicalName=true;
jdbcPrincipal=hive/_HOST@GCE.CLOUDERA.COM
examplesRoot=/testproject/code/examplesoozie

oozie.use.system.libpath=true

oozie.wf.application.path=${nameNode}${examplesRoot}/hive2
[root@vjd-1 hive2]# cat workflow.xml 
<workflow-app xmlns="uri:oozie:workflow:0.5" name="hive2-wf">
    <credentials>
        <credential name="hs2-creds" type="hive2">
            <property>
                <name>hive2.server.principal</name>
                <value>${jdbcPrincipal}</value>
            </property>
            <property>
                <name>hive2.jdbc.url</name>
                <value>${jdbcURL}</value>
            </property>
        </credential>
    </credentials>
    <start to="hive2-node"/>

    <action name="hive2-node"  cred="hs2-creds">
        <hive2 xmlns="uri:oozie:hive2-action:0.1">
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <prepare>
                <delete path="${nameNode}${examplesRoot}/output-data/hive2"/>
                <mkdir path="${nameNode}${examplesRoot}/output-data"/>
            </prepare>
            <configuration>
                <property>
                    <name>mapred.job.queue.name</name>
                    <value>${queueName}</value>
                </property>
            </configuration>
            <jdbc-url>${jdbcURL}</jdbc-url>
            <script>script.q</script>
            <param>INPUT=${examplesRoot}/input-data/table</param>
            <param>OUTPUT=${examplesRoot}/output-data/hive2</param>
        </hive2>
        <ok to="end"/>
        <error to="fail"/>
    </action>

    <kill name="fail">
        <message>Hive2 (Beeline) action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end"/>
</workflow-app>
[root@vjd-1 hive2]# oozie job -oozie http://vjtest-local.gce.cloudera.com:11000/oozie -config
hive2job.properties -run
job: 0000041-160522222507746-oozie-oozi-W
[root@vjd-1 hive2]#  oozie job -oozie http://vjtest-local.gce.cloudera.com:11000/oozie -info
0000041-160522222507746-oozie-oozi-W 
Job ID : 0000041-160522222507746-oozie-oozi-W
------------------------------------------------------------------------------------------------------------------------------------
Workflow Name : hive2-wf
App Path      : hdfs://vjd-1.gce.cloudera.com:8020/testproject/code/examplesoozie/hive2
Status        : SUCCEEDED
Run           : 0
User          : vijay
Group         : -
Created       : 2016-05-23 05:38 GMT
Started       : 2016-05-23 05:38 GMT
Last Modified : 2016-05-23 05:38 GMT
Ended         : 2016-05-23 05:38 GMT
CoordAction ID: -

Actions
------------------------------------------------------------------------------------------------------------------------------------
ID                                                                            Status    Ext
ID                 Ext Status Err Code  
------------------------------------------------------------------------------------------------------------------------------------
0000041-160522222507746-oozie-oozi-W@:start:                                  OK        -
                     OK         -         
------------------------------------------------------------------------------------------------------------------------------------
0000041-160522222507746-oozie-oozi-W@hive2-node                               OK        job_1463802572747_0018
SUCCEEDED  -         
------------------------------------------------------------------------------------------------------------------------------------
0000041-160522222507746-oozie-oozi-W@end                                      OK        -
                     OK         -         
------------------------------------------------------------------------------------------------------------------------------------

[root@vjd-1 hive2]# cat script.q
DROP TABLE IF EXISTS test;
SELECT * FROM sample_07;
[root@vjd-1 hive2]# 
{code}

> Allow DNS CNAME ALIAS Resolution from apache hive beeline JDBC URL to allow for failover
> ----------------------------------------------------------------------------------------
>
>                 Key: HIVE-13817
>                 URL: https://issues.apache.org/jira/browse/HIVE-13817
>             Project: Hive
>          Issue Type: New Feature
>          Components: Beeline
>    Affects Versions: 1.2.1
>            Reporter: Vijay Singh
>         Attachments: HIVE-13817.1.patch, HIVE-13817.2.patch, HIVE-13817.3.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Currently, in case of BDR clusters, DNS CNAME alias based connections fail. As _HOST
resolves to exact endpoint specified in connection string and that may not be intended SPN
for kerberos based on reverse DNS lookup. Consequently this JIRA proposes that client specific
setting be used to resolv _HOST from CNAME DNS alias to A record entry on the fly in beeline.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message