hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-13896) distribution tarball is missing almost all of the hadoop jars
Date Wed, 04 Jan 2017 01:21:58 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-13896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andrew Wang updated HADOOP-13896:
---------------------------------
    Attachment: hadoop-13896.001.patch

A little more progress debugging this, the {{src}} profile is causing the per-component assembly
plugin invocation to run before the jars are created. I ran a {{dist}} and a {{dist,src}}
build, and then grepped as follows: {{egrep "(maven-assembly-plugin|maven-jar-plugin)" dist.out
| grep hadoop-common}}

dist:

{noformat}
[INFO] --- maven-jar-plugin:2.5:jar (prepare-jar) @ hadoop-common ---
[INFO] --- maven-jar-plugin:2.5:test-jar (prepare-test-jar) @ hadoop-common ---
[INFO] --- maven-assembly-plugin:2.4:single (dist) @ hadoop-common ---
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ hadoop-common ---
{noformat}

dist,src:

{noformat}
[INFO] --- maven-assembly-plugin:2.4:single (dist) @ hadoop-common ---
[INFO] --- maven-jar-plugin:2.5:jar (prepare-jar) @ hadoop-common ---
[INFO] --- maven-jar-plugin:2.5:test-jar (prepare-test-jar) @ hadoop-common ---
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ hadoop-common ---
{noformat}

Here's a patch that just changes the dist assembly plugin in "package" instead of "prepare-package",
which fixes the ordering problem. I did a build and diffed the jars in alpha1 against trunk
with this patch applied, and only found the changed lib dependencies and the new shaded client
and aliyun JARs.

> distribution tarball is missing almost all of the hadoop jars
> -------------------------------------------------------------
>
>                 Key: HADOOP-13896
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13896
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: build
>    Affects Versions: 3.0.0-alpha2
>            Reporter: Allen Wittenauer
>            Assignee: Andrew Wang
>            Priority: Blocker
>         Attachments: hadoop-13896.001.patch
>
>
> From what I can tell, all of the hdfs and common jars from their respective lib dirs
are missing, excluding hadoop-hdfs-client and hadoop-hdfs-nfs. But there are likely more.
> Steps to reproduce:
> 1. ./start-build-env.sh
> 2. mvn install -Pdist,src -Dtar -DskipTests -Pnative
> 3. ls hadoop-dist/target/hadoop-3.0.0-alpha2-SNAPSHOT/share/hadoop/common
> hadoop-nfs-3.0.0-alpha2-SNAPSHOT.jar  jdiff  lib  templates  webapps
> 4. ls hadoop-dist/target/hadoop-3.0.0-alpha2-SNAPSHOT/share/hadoop/hdfs
> hadoop-hdfs-nfs-3.0.0-alpha2-SNAPSHOT.jar  jdiff  lib  templates  webapps



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message