spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zineng Yuan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-25061) Spark SQL Thrift Server fails to not pick up hiveconf passing parameter
Date Wed, 08 Aug 2018 21:31:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-25061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573911#comment-16573911
] 

Zineng Yuan commented on SPARK-25061:
-------------------------------------

A further investigation starting from thrift server source points out that the problem comes
from HiveClientImpl( Line 154):: after HiveConf is created (with hiveconf property), these
confs got overwritten by hadooConf object (with hive-site.xml).  

[https://github.com/apache/spark/blob/branch-2.2/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala]

[https://github.com/apache/spark/blob/branch-2.2/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2.scala]

 Don't understand why passing parameters got overwritten by hive-site.xml. Passing parameters
should override others.

 

 

>  Spark SQL Thrift Server fails to not pick up hiveconf passing parameter
> ------------------------------------------------------------------------
>
>                 Key: SPARK-25061
>                 URL: https://issues.apache.org/jira/browse/SPARK-25061
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.2.0
>            Reporter: Zineng Yuan
>            Priority: Critical
>
> Spark thrift server should pick up passing parameter value and overwrites the conf value 
rom hive-site.xml. For example,  the server should overwrite what exists in hive-site.xml.
>  ./sbin/start-thriftserver.sh --master yarn-client ...
> --hiveconf "hive.server2.authentication.kerberos.principal=<client_principal>"
...
> <property>
> <name>hive.server2.authentication.kerberos.principal</name>
> <value>hive/_HOST@<server_domain></value>
> </property>
> However, the server takes what in hive-site.xml.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message