hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward Capriolo (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-3017) hive-exec jar, contains classes from other modules(hive-serde, hive-shims, hive-common etc) duplicating those classes in two jars
Date Sun, 18 Nov 2012 00:44:12 GMT

    [ https://issues.apache.org/jira/browse/HIVE-3017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13499569#comment-13499569
] 

Edward Capriolo commented on HIVE-3017:
---------------------------------------

It is more then just a fat jar issue. If you are living in the same class loader you can only
load a class once. So if hive-ql uses common-util-X.Y.Z and some other piece of code uses
commun-util-X.Y.G and there is a breaking change in common-util you can not satisfy that.


Now as to the fat jar issue. You have a choice fat jar or use hadoops -distjar. Either produces
the same result all the jars are on the classpath. 

I am not saying there is no way out of this problem, but it is like this purposely. Other
solutions have been proposed but there is no follow through. Some other tickets talk about
this but I do not have the number off hand.
                
> hive-exec jar, contains classes from other modules(hive-serde, hive-shims, hive-common
etc) duplicating those classes in two jars
> ---------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3017
>                 URL: https://issues.apache.org/jira/browse/HIVE-3017
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Jakob Homan
>
> HIVE-2646 added the jars from hive-serde to the hive-exec class:
> {noformat}
> ...
>      0 Wed May 09 20:56:30 PDT 2012 org/apache/hadoop/hive/serde2/typeinfo/
>   1971 Wed May 09 20:56:28 PDT 2012 org/apache/hadoop/hive/serde2/typeinfo/ListTypeInfo.class
>   2396 Wed May 09 20:56:28 PDT 2012 org/apache/hadoop/hive/serde2/typeinfo/MapTypeInfo.class
>   2788 Wed May 09 20:56:28 PDT 2012 org/apache/hadoop/hive/serde2/typeinfo/PrimitiveTypeInfo.class
>   4408 Wed May 09 20:56:28 PDT 2012 org/apache/hadoop/hive/serde2/typeinfo/StructTypeInfo.class
>    900 Wed May 09 20:56:28 PDT 2012 org/apache/hadoop/hive/serde2/typeinfo/TypeInfo.class
>   6576 Wed May 09 20:56:28 PDT 2012 org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.class
>   1231 Wed May 09 20:56:28 PDT 2012 org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils$1.class
>   1239 Wed May 09 20:56:28 PDT 2012 org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils$TypeInfoParser$Token.class
>   7145 Wed May 09 20:56:28 PDT 2012 org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils$TypeInfoParser.class
>  14482 Wed May 09 20:56:28 PDT 2012 org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils.class
>   2594 Wed May 09 20:56:28 PDT 2012 org/apache/hadoop/hive/serde2/typeinfo/UnionTypeInfo.class
>    144 Wed May 09 20:56:30 PDT 2012 org/apache/hadoop/hive/serde2/typeinfo/package-info.class
> ...{noformat}
> Was this intentional? If so, the serde jar should be deprecated. If not, the serde classes
should be removed since this creates two sources of truth for them and can cause other problems
(see HCATALOG-407).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message