hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuefu Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-12179) Add option to not add spark-assembly.jar to Hive classpath
Date Wed, 14 Oct 2015 21:59:05 GMT

    [ https://issues.apache.org/jira/browse/HIVE-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14957890#comment-14957890
] 

Xuefu Zhang commented on HIVE-12179:
------------------------------------

I think the option should be another way around: not adding it if a flag is set. Otherwise,
existing users will have to set the flag in order to get the existing behavior.

> Add option to not add spark-assembly.jar to Hive classpath
> ----------------------------------------------------------
>
>                 Key: HIVE-12179
>                 URL: https://issues.apache.org/jira/browse/HIVE-12179
>             Project: Hive
>          Issue Type: Bug
>          Components: Spark
>            Reporter: Jason Dere
>            Assignee: Jason Dere
>         Attachments: HIVE-12179.1.patch
>
>
> After running the following Hive script:
> {noformat}
> add jar hdfs:///tmp/junit-4.11.jar;
> show tables;
> {noformat}
> I can see the following lines getting printed to stdout when Hive exits:
> {noformat}
> WARN: The method class org.apache.commons.logging.impl.SLF4JLogFactory#release() was
invoked.
> WARN: Please see http://www.slf4j.org/codes.html#release for an explanation.
> {noformat}
> Also seeing the following warnings in stderr:
> {noformat}
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in [jar:file:/usr/hdp/2.3.3.0-2981/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/usr/hdp/2.3.3.0-2981/spark/lib/spark-assembly-1.4.1.2.3.3.0-2981-hadoop2.7.1.2.3.3.0-2981.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in [jar:file:/usr/hdp/2.3.3.0-2981/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/usr/hdp/2.3.3.0-2981/spark/lib/spark-assembly-1.4.1.2.3.3.0-2981-hadoop2.7.1.2.3.3.0-2981.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> {noformat}
> It looks like this is due to the addition of the shaded spark-assembly.jar to the classpath,
which contains classes from icl-over-slf4j.jar (which is causing the stdout messages) and
slf4j-log4j12.jar.
> Removing spark-assembly.jar from being added to the classpath causes these messages to
go away. It would be good to have a way to specify that Hive not add spark-assembly.jar to
the class path.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message