hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Mackrory (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-13952) tools dependency hooks are throwing errors
Date Fri, 04 Aug 2017 19:52:03 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-13952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sean Mackrory updated HADOOP-13952:
-----------------------------------
    Attachment: HADOOP-13952.000.patch

I added an "extra_dir" parameter so you can specify a dependency. It's used to specify that
MapReduce can rely on whatever is already in YARN's classpath as well as the common stuff.
That cut down most of the duplicated JARs. What's left are the okio and okhttp JARs and just
a couple of things that happen to also be in HDFS.

Very slight increase in bloat, but I feel this is the more correct solution. If anyone wants
to argue that just accepting it anywhere under ./share and minimizing duplication is better,
I'd be happy to create and test that patch too, but this is what I'm proposing for inclusion
to resolve this issue.

> tools dependency hooks are throwing errors
> ------------------------------------------
>
>                 Key: HADOOP-13952
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13952
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: build
>    Affects Versions: 3.0.0-alpha2
>            Reporter: Allen Wittenauer
>            Assignee: Sean Mackrory
>            Priority: Critical
>         Attachments: HADOOP-13952.000.patch, HADOOP-13952.preview.patch
>
>
> During build, we are throwing these errors:
> {code}
> ERROR: hadoop-aliyun has missing dependencies: jasper-compiler-5.5.23.jar
> ERROR: hadoop-aliyun has missing dependencies: json-lib-jdk15.jar
> ERROR: hadoop-archive-logs has missing dependencies: jasper-compiler-5.5.23.jar
> ERROR: hadoop-archives has missing dependencies: jasper-compiler-5.5.23.jar
> ERROR: hadoop-aws has missing dependencies: jasper-compiler-5.5.23.jar
> ERROR: hadoop-azure has missing dependencies: jetty-util-ajax-9.3.11.v20160721.jar
> ERROR: hadoop-azure-datalake has missing dependencies: okhttp-2.4.0.jar
> ERROR: hadoop-azure-datalake has missing dependencies: okio-1.4.0.jar
> ERROR: hadoop-extras has missing dependencies: jasper-compiler-5.5.23.jar
> ERROR: hadoop-gridmix has missing dependencies: jasper-compiler-5.5.23.jar
> ERROR: hadoop-kafka has missing dependencies: lz4-1.2.0.jar
> ERROR: hadoop-kafka has missing dependencies: kafka-clients-0.8.2.1.jar
> ERROR: hadoop-openstack has missing dependencies: commons-httpclient-3.1.jar
> ERROR: hadoop-rumen has missing dependencies: jasper-compiler-5.5.23.jar
> ERROR: hadoop-sls has missing dependencies: jasper-compiler-5.5.23.jar
> ERROR: hadoop-sls has missing dependencies: metrics-core-3.0.1.jar
> ERROR: hadoop-streaming has missing dependencies: jasper-compiler-5.5.23.jar
> {code}
> Likely a variety of reasons for the failures.  Kafka is HADOOP-12556, but others need
to be investigated.  Probably just need to look at more than just common/lib in dist-tools-hooks-maker
now that shading has gone in.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message