hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-18861) druid-hdfs-storage is pulling in hadoop-aws-2.7.x and aws SDK, creating classpath problems on hadoop 3.x
Date Mon, 05 Mar 2018 15:44:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-18861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16386231#comment-16386231
] 

Steve Loughran commented on HIVE-18861:
---------------------------------------

And after
{code}
[INFO] |  +- io.druid.extensions:druid-hdfs-storage:jar:0.11.0:compile
[INFO] |  |  \- (org.apache.hadoop:hadoop-client:jar:3.1.0-SNAPSHOT:compile - version managed
from 2.7.3; omitted for duplicate)
{code}


> druid-hdfs-storage is pulling in hadoop-aws-2.7.x and aws SDK, creating classpath problems
on hadoop 3.x
> --------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-18861
>                 URL: https://issues.apache.org/jira/browse/HIVE-18861
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Druid integration
>    Affects Versions: 3.0.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>            Priority: Major
>         Attachments: HIVE-18861-001.patch
>
>
> druid-hdfs-storage JAR is transitively pulling in hadoop-aws JAR 2.7.3, which creates
classpath problems as a set of aws-sdk 1.10.77 JARs get on the CP, even with Hadoop 3 &
its move to a full aws-sdk-bundle JAR.
> Two options
> # exclude the dependency
> # force it up to whatever ${hadoop.version} is, so make it consistent



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message