hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Carl Steinbach (Commented) (JIRA)" <>
Subject [jira] [Commented] (HIVE-2646) Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
Date Thu, 12 Apr 2012 22:29:19 GMT


Carl Steinbach commented on HIVE-2646:

Here's the exception stack from the TestLazySimpleSerDe test failure:

    [junit] Running org.apache.hadoop.hive.serde2.lazy.TestLazySimpleSerDe
    [junit] java.lang.ArrayIndexOutOfBoundsException: -65
    [junit] 	at org.apache.commons.codec.binary.Base64.isBase64(
    [junit] 	at org.apache.commons.codec.binary.Base64.isArrayByteBase64(
    [junit] 	at org.apache.hadoop.hive.serde2.lazy.LazyBinary.init(
    [junit] 	at org.apache.hadoop.hive.serde2.lazy.LazyStruct.uncheckedGetField(
    [junit] 	at org.apache.hadoop.hive.serde2.lazy.LazyStruct.getField(
    [junit] 	at org.apache.hadoop.hive.serde2.lazy.objectinspector.LazySimpleStructObjectInspector.getStructFieldData(
    [junit] 	at org.apache.hadoop.hive.serde2.lazy.TestLazySimpleSerDe.deserializeAndSerialize(
    [junit] 	at org.apache.hadoop.hive.serde2.lazy.TestLazySimpleSerDe.testLazySimpleSerDe(

The ArrayIndexOutOfBoundsException thrown by Base64.isBase64() is due to a bug in commons-codec
1.3. This was fixed in CODEC-22 in commons-codec 1.4. Right now the test classpath contains
copies of versions 1.3 and 1.4 of commons-codec, and the version 1.3 JAR precedes version
1.4. Ideally we can prevent the 1.3 JAR from getting pulled down entirely, but at the least
we need to make sure it appears on the classpath after the v1.4 JAR.

@Andrew: can you please look into this? Thanks.
> Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
> ----------------------------------------------------------------------------
>                 Key: HIVE-2646
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: Build Infrastructure
>    Affects Versions: 0.8.0
>            Reporter: Andrew Bayer
>            Assignee: Andrew Bayer
>            Priority: Critical
>             Fix For: 0.9.0
>         Attachments: HIVE-2646.D2133.1.patch, HIVE-2646.D2133.10.patch, HIVE-2646.D2133.11.patch,
HIVE-2646.D2133.12.patch, HIVE-2646.D2133.13.patch, HIVE-2646.D2133.14.patch, HIVE-2646.D2133.2.patch,
HIVE-2646.D2133.3.patch, HIVE-2646.D2133.4.patch, HIVE-2646.D2133.5.patch, HIVE-2646.D2133.6.patch,
HIVE-2646.D2133.7.patch, HIVE-2646.D2133.8.patch, HIVE-2646.D2133.9.patch, HIVE-2646.diff.txt
> The current Hive Ivy dependency logic for its Hadoop dependencies is problematic - depending
on the tarball and extracting the jars from there, rather than depending on the jars directly.
It'd be great if this was fixed to actually have the jar dependencies defined directly.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message