hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bharat Viswanadham (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HADOOP-14672) Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, dom, etc.
Date Mon, 24 Jul 2017 22:08:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099179#comment-16099179
] 

Bharat Viswanadham edited comment on HADOOP-14672 at 7/24/17 10:07 PM:
-----------------------------------------------------------------------

[~busbey]
I tried your patch. below is the output.  testJar,testShell classes are from hadoop only (hadoop-mapreduce-client-jobclient)
test. 
So, for this I think these should not be shaded. 

 Found artifact with unexpected contents: '/Users/bviswanadham/workspace/hadoop/hadoop-client-modules/hadoop-client-minicluster/target/hadoop-client-minicluster-3.0.0-beta1-SNAPSHOT.jar'
    Please check the following and either correct the build or update
    the allowed list with reasoning.

    capacity-scheduler.xml
    krb5.conf
    log4j.properties
    about.html
    testjar/
    testjar/ClassWordCount$MapClass.class
    testjar/ClassWordCount$Reduce.class
    testjar/ClassWordCount.class
    testjar/CustomOutputCommitter.class
    testjar/ExternalIdentityReducer.class
    testjar/ExternalMapperReducer.class
    testjar/ExternalWritable.class
    testjar/JobKillCommitter$CommitterWithFailCleanup.class
    testjar/JobKillCommitter$CommitterWithFailSetup.class
    testjar/JobKillCommitter$CommitterWithNoError.class
    testjar/JobKillCommitter$MapperFail.class
    testjar/JobKillCommitter$MapperPass.class
    testjar/JobKillCommitter$MapperPassSleep.class
    testjar/JobKillCommitter$ReducerFail.class
    testjar/JobKillCommitter$ReducerPass.class
    testjar/JobKillCommitter.class
    testjar/UserNamePermission$UserNameMapper.class
    testjar/UserNamePermission$UserNameReducer.class
    testjar/UserNamePermission.class
    testshell/
    testshell/ExternalMapReduce$MapClass.class
    testshell/ExternalMapReduce$Reduce.class
    testshell/ExternalMapReduce.class
    .options
    jdtCompilerAdapter.jar
    plugin.properties
    plugin.xml
    container-log4j.properties
    java.policy
    catalog.cat
    javaee_5.xsd
    javaee_6.xsd
    javaee_web_services_client_1_2.xsd
    javaee_web_services_client_1_3.xsd
    jsp_2_1.xsd
    jsp_2_2.xsd
    web-app_2_5.xsd
    web-app_3_0.xsd
    web-common_3_0.xsd
    xml.xsd



was (Author: bharatviswa):
[~busbey]
I tried your patch. below is the output.  testJar classes are from hadoop only (hadoop-mapreduce-client-jobclient)
test. 
So, for this I think these should not be shaded. 

 Found artifact with unexpected contents: '/Users/bviswanadham/workspace/hadoop/hadoop-client-modules/hadoop-client-minicluster/target/hadoop-client-minicluster-3.0.0-beta1-SNAPSHOT.jar'
    Please check the following and either correct the build or update
    the allowed list with reasoning.

    capacity-scheduler.xml
    krb5.conf
    log4j.properties
    about.html
    testjar/
    testjar/ClassWordCount$MapClass.class
    testjar/ClassWordCount$Reduce.class
    testjar/ClassWordCount.class
    testjar/CustomOutputCommitter.class
    testjar/ExternalIdentityReducer.class
    testjar/ExternalMapperReducer.class
    testjar/ExternalWritable.class
    testjar/JobKillCommitter$CommitterWithFailCleanup.class
    testjar/JobKillCommitter$CommitterWithFailSetup.class
    testjar/JobKillCommitter$CommitterWithNoError.class
    testjar/JobKillCommitter$MapperFail.class
    testjar/JobKillCommitter$MapperPass.class
    testjar/JobKillCommitter$MapperPassSleep.class
    testjar/JobKillCommitter$ReducerFail.class
    testjar/JobKillCommitter$ReducerPass.class
    testjar/JobKillCommitter.class
    testjar/UserNamePermission$UserNameMapper.class
    testjar/UserNamePermission$UserNameReducer.class
    testjar/UserNamePermission.class
    testshell/
    testshell/ExternalMapReduce$MapClass.class
    testshell/ExternalMapReduce$Reduce.class
    testshell/ExternalMapReduce.class
    .options
    jdtCompilerAdapter.jar
    plugin.properties
    plugin.xml
    container-log4j.properties
    java.policy
    catalog.cat
    javaee_5.xsd
    javaee_6.xsd
    javaee_web_services_client_1_2.xsd
    javaee_web_services_client_1_3.xsd
    jsp_2_1.xsd
    jsp_2_2.xsd
    web-app_2_5.xsd
    web-app_3_0.xsd
    web-common_3_0.xsd
    xml.xsd


> Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, dom, etc.
> --------------------------------------------------------------------------------------
>
>                 Key: HADOOP-14672
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14672
>             Project: Hadoop Common
>          Issue Type: Sub-task
>            Reporter: Junping Du
>            Assignee: Bharat Viswanadham
>            Priority: Blocker
>         Attachments: HADOOP-14672.02.patch, HADOOP-14672.03.patch, HADOOP-14672.04.patch,
HADOOP-14672.patch
>
>
> The shaded hadoop-client-minicluster shouldn't include any unshaded dependencies, but
we can see: javax, dom, sax, etc. are all unshaded.
> CC [~busbey]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message