hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bharat Viswanadham (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HADOOP-14672) Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, dom, etc.
Date Fri, 21 Jul 2017 23:25:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16096989#comment-16096989
] 

Bharat Viswanadham edited comment on HADOOP-14672 at 7/21/17 11:24 PM:
-----------------------------------------------------------------------

Hi Sean,
It is pulled from hadoop-minicluster. xml-apis' is coming from xerces:xercesImpl. 
I have seen in hadoop-client-runtime, you have filtered it.
I am not sure, can i do the same here.
<filter>
                      <artifact>xerces:xercesImpl</artifact>
                      <excludes>
                        <exclude>META-INF/services/*</exclude>
                      </excludes>
Pls let me know your thoughts on this?

Maven Dependency Tree:
[INFO] --- maven-dependency-plugin:2.10:tree (default-cli) @ hadoop-minicluster ---
[INFO] org.apache.hadoop:hadoop-minicluster:jar:3.0.0-beta1-SNAPSHOT
[INFO] +- org.apache.hadoop:hadoop-common:test-jar:tests:3.0.0-beta1-SNAPSHOT:compile
[INFO] |  +- org.apache.hadoop:hadoop-annotations:jar:3.0.0-beta1-SNAPSHOT:compile
[INFO] |  |  \- jdk.tools:jdk.tools:jar:1.8:system
[INFO] |  +- com.google.guava:guava:jar:11.0.2:compile
[INFO] |  +- commons-cli:commons-cli:jar:1.2:compile
[INFO] |  +- org.apache.commons:commons-math3:jar:3.1.1:compile
[INFO] |  +- org.apache.httpcomponents:httpclient:jar:4.5.2:compile
[INFO] |  |  \- org.apache.httpcomponents:httpcore:jar:4.4.4:compile
[INFO] |  +- commons-codec:commons-codec:jar:1.4:compile
[INFO] |  +- commons-io:commons-io:jar:2.4:compile
[INFO] |  +- commons-net:commons-net:jar:3.1:compile
[INFO] |  +- commons-collections:commons-collections:jar:3.2.2:compile
[INFO] |  +- javax.servlet:javax.servlet-api:jar:3.1.0:compile
[INFO] |  +- org.eclipse.jetty:jetty-server:jar:9.3.11.v20160721:compile
[INFO] |  |  +- org.eclipse.jetty:jetty-http:jar:9.3.11.v20160721:compile
[INFO] |  |  \- org.eclipse.jetty:jetty-io:jar:9.3.11.v20160721:compile
[INFO] |  +- org.eclipse.jetty:jetty-util:jar:9.3.11.v20160721:compile
[INFO] |  +- org.eclipse.jetty:jetty-servlet:jar:9.3.11.v20160721:compile
[INFO] |  |  \- org.eclipse.jetty:jetty-security:jar:9.3.11.v20160721:compile
[INFO] |  +- org.eclipse.jetty:jetty-webapp:jar:9.3.11.v20160721:compile
[INFO] |  |  \- org.eclipse.jetty:jetty-xml:jar:9.3.11.v20160721:compile
[INFO] |  +- javax.servlet.jsp:jsp-api:jar:2.1:runtime
[INFO] |  +- com.sun.jersey:jersey-core:jar:1.19:compile
[INFO] |  |  \- javax.ws.rs:jsr311-api:jar:1.1.1:compile
[INFO] |  +- com.sun.jersey:jersey-servlet:jar:1.19:compile
[INFO] |  +- com.sun.jersey:jersey-json:jar:1.19:compile
[INFO] |  |  +- org.codehaus.jettison:jettison:jar:1.1:compile
[INFO] |  |  +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile
[INFO] |  |  +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] |  |  +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO] |  |  +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.13:compile
[INFO] |  |  \- org.codehaus.jackson:jackson-xc:jar:1.9.13:compile
[INFO] |  +- com.sun.jersey:jersey-server:jar:1.19:compile
[INFO] |  +- commons-logging:commons-logging:jar:1.1.3:compile
[INFO] |  +- log4j:log4j:jar:1.2.17:compile
[INFO] |  +- commons-lang:commons-lang:jar:2.6:compile
[INFO] |  +- commons-beanutils:commons-beanutils:jar:1.9.3:compile
[INFO] |  +- org.apache.commons:commons-configuration2:jar:2.1:compile
[INFO] |  |  \- org.apache.commons:commons-lang3:jar:3.3.2:compile
[INFO] |  +- org.slf4j:slf4j-api:jar:1.7.25:compile
[INFO] |  +- org.slf4j:slf4j-log4j12:jar:1.7.25:compile
[INFO] |  +- org.apache.avro:avro:jar:1.7.4:compile
[INFO] |  |  +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile
[INFO] |  |  \- org.xerial.snappy:snappy-java:jar:1.0.4.1:compile
[INFO] |  +- com.google.re2j:re2j:jar:1.0:compile
[INFO] |  +- com.google.protobuf:protobuf-java:jar:2.5.0:compile
[INFO] |  +- com.google.code.gson:gson:jar:2.2.4:compile
[INFO] |  +- org.apache.hadoop:hadoop-auth:jar:3.0.0-beta1-SNAPSHOT:compile
[INFO] |  |  +- com.nimbusds:nimbus-jose-jwt:jar:3.9:compile
[INFO] |  |  |  +- net.jcip:jcip-annotations:jar:1.0:compile
[INFO] |  |  |  \- net.minidev:json-smart:jar:1.1.1:compile
[INFO] |  |  \- org.apache.curator:curator-framework:jar:2.12.0:compile
[INFO] |  +- com.jcraft:jsch:jar:0.1.54:compile
[INFO] |  +- org.apache.curator:curator-client:jar:2.12.0:compile
[INFO] |  +- org.apache.curator:curator-recipes:jar:2.12.0:compile
[INFO] |  +- com.google.code.findbugs:jsr305:jar:3.0.0:compile
[INFO] |  +- org.apache.htrace:htrace-core4:jar:4.1.0-incubating:compile
[INFO] |  +- org.apache.zookeeper:zookeeper:jar:3.4.9:compile
[INFO] |  +- org.apache.commons:commons-compress:jar:1.4.1:compile
[INFO] |  |  \- org.tukaani:xz:jar:1.0:compile
[INFO] |  +- org.apache.kerby:kerb-simplekdc:jar:1.0.0:compile
[INFO] |  |  +- org.apache.kerby:kerb-client:jar:1.0.0:compile
[INFO] |  |  |  +- org.apache.kerby:kerby-config:jar:1.0.0:compile
[INFO] |  |  |  +- org.apache.kerby:kerb-core:jar:1.0.0:compile
[INFO] |  |  |  |  \- org.apache.kerby:kerby-pkix:jar:1.0.0:compile
[INFO] |  |  |  |     +- org.apache.kerby:kerby-asn1:jar:1.0.0:compile
[INFO] |  |  |  |     \- org.apache.kerby:kerby-util:jar:1.0.0:compile
[INFO] |  |  |  +- org.apache.kerby:kerb-common:jar:1.0.0:compile
[INFO] |  |  |  |  \- org.apache.kerby:kerb-crypto:jar:1.0.0:compile
[INFO] |  |  |  \- org.apache.kerby:kerb-util:jar:1.0.0:compile
[INFO] |  |  \- org.apache.kerby:kerb-admin:jar:1.0.0:compile
[INFO] |  |     +- org.apache.kerby:kerb-server:jar:1.0.0:compile
[INFO] |  |     |  \- org.apache.kerby:kerb-identity:jar:1.0.0:compile
[INFO] |  |     \- org.apache.kerby:kerby-xdr:jar:1.0.0:compile
[INFO] |  +- com.fasterxml.jackson.core:jackson-databind:jar:2.7.8:compile
[INFO] |  |  \- com.fasterxml.jackson.core:jackson-core:jar:2.7.8:compile
[INFO] |  +- org.codehaus.woodstox:stax2-api:jar:3.1.4:compile
[INFO] |  \- com.fasterxml.woodstox:woodstox-core:jar:5.0.3:compile
[INFO] +- org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0-beta1-SNAPSHOT:compile
[INFO] |  +- org.eclipse.jetty:jetty-util-ajax:jar:9.3.11.v20160721:compile
[INFO] |  +- commons-daemon:commons-daemon:jar:1.0.13:compile
[INFO] |  +- io.netty:netty:jar:3.10.5.Final:compile
[INFO] |  +- io.netty:netty-all:jar:4.0.23.Final:compile
*[INFO] |  +- xerces:xercesImpl:jar:2.9.1:compile
[INFO] |  |  \- xml-apis:xml-apis:jar:1.3.04:compile*


was (Author: bharatviswa):
Hi Sean,
It is pulled from hadoop-minicluster. xml-apis' is coming from xerces:xercesImpl. 
I have seen in hadoop-client-runtime, you have filtered it.
I am not sure, can i do the same here.
<filter>
                      <artifact>xerces:xercesImpl</artifact>
                      <excludes>
                        <exclude>META-INF/services/*</exclude>
                      </excludes>
Pls let me know your thoughts on this?

Maven Dependency Tree:
[INFO] --- maven-dependency-plugin:2.10:tree (default-cli) @ hadoop-minicluster ---
[INFO] org.apache.hadoop:hadoop-minicluster:jar:3.0.0-beta1-SNAPSHOT
[INFO] +- org.apache.hadoop:hadoop-common:test-jar:tests:3.0.0-beta1-SNAPSHOT:compile
[INFO] |  +- org.apache.hadoop:hadoop-annotations:jar:3.0.0-beta1-SNAPSHOT:compile
[INFO] |  |  \- jdk.tools:jdk.tools:jar:1.8:system
[INFO] |  +- com.google.guava:guava:jar:11.0.2:compile
[INFO] |  +- commons-cli:commons-cli:jar:1.2:compile
[INFO] |  +- org.apache.commons:commons-math3:jar:3.1.1:compile
[INFO] |  +- org.apache.httpcomponents:httpclient:jar:4.5.2:compile
[INFO] |  |  \- org.apache.httpcomponents:httpcore:jar:4.4.4:compile
[INFO] |  +- commons-codec:commons-codec:jar:1.4:compile
[INFO] |  +- commons-io:commons-io:jar:2.4:compile
[INFO] |  +- commons-net:commons-net:jar:3.1:compile
[INFO] |  +- commons-collections:commons-collections:jar:3.2.2:compile
[INFO] |  +- javax.servlet:javax.servlet-api:jar:3.1.0:compile
[INFO] |  +- org.eclipse.jetty:jetty-server:jar:9.3.11.v20160721:compile
[INFO] |  |  +- org.eclipse.jetty:jetty-http:jar:9.3.11.v20160721:compile
[INFO] |  |  \- org.eclipse.jetty:jetty-io:jar:9.3.11.v20160721:compile
[INFO] |  +- org.eclipse.jetty:jetty-util:jar:9.3.11.v20160721:compile
[INFO] |  +- org.eclipse.jetty:jetty-servlet:jar:9.3.11.v20160721:compile
[INFO] |  |  \- org.eclipse.jetty:jetty-security:jar:9.3.11.v20160721:compile
[INFO] |  +- org.eclipse.jetty:jetty-webapp:jar:9.3.11.v20160721:compile
[INFO] |  |  \- org.eclipse.jetty:jetty-xml:jar:9.3.11.v20160721:compile
[INFO] |  +- javax.servlet.jsp:jsp-api:jar:2.1:runtime
[INFO] |  +- com.sun.jersey:jersey-core:jar:1.19:compile
[INFO] |  |  \- javax.ws.rs:jsr311-api:jar:1.1.1:compile
[INFO] |  +- com.sun.jersey:jersey-servlet:jar:1.19:compile
[INFO] |  +- com.sun.jersey:jersey-json:jar:1.19:compile
[INFO] |  |  +- org.codehaus.jettison:jettison:jar:1.1:compile
[INFO] |  |  +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile
[INFO] |  |  +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] |  |  +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO] |  |  +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.13:compile
[INFO] |  |  \- org.codehaus.jackson:jackson-xc:jar:1.9.13:compile
[INFO] |  +- com.sun.jersey:jersey-server:jar:1.19:compile
[INFO] |  +- commons-logging:commons-logging:jar:1.1.3:compile
[INFO] |  +- log4j:log4j:jar:1.2.17:compile
[INFO] |  +- commons-lang:commons-lang:jar:2.6:compile
[INFO] |  +- commons-beanutils:commons-beanutils:jar:1.9.3:compile
[INFO] |  +- org.apache.commons:commons-configuration2:jar:2.1:compile
[INFO] |  |  \- org.apache.commons:commons-lang3:jar:3.3.2:compile
[INFO] |  +- org.slf4j:slf4j-api:jar:1.7.25:compile
[INFO] |  +- org.slf4j:slf4j-log4j12:jar:1.7.25:compile
[INFO] |  +- org.apache.avro:avro:jar:1.7.4:compile
[INFO] |  |  +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile
[INFO] |  |  \- org.xerial.snappy:snappy-java:jar:1.0.4.1:compile
[INFO] |  +- com.google.re2j:re2j:jar:1.0:compile
[INFO] |  +- com.google.protobuf:protobuf-java:jar:2.5.0:compile
[INFO] |  +- com.google.code.gson:gson:jar:2.2.4:compile
[INFO] |  +- org.apache.hadoop:hadoop-auth:jar:3.0.0-beta1-SNAPSHOT:compile
[INFO] |  |  +- com.nimbusds:nimbus-jose-jwt:jar:3.9:compile
[INFO] |  |  |  +- net.jcip:jcip-annotations:jar:1.0:compile
[INFO] |  |  |  \- net.minidev:json-smart:jar:1.1.1:compile
[INFO] |  |  \- org.apache.curator:curator-framework:jar:2.12.0:compile
[INFO] |  +- com.jcraft:jsch:jar:0.1.54:compile
[INFO] |  +- org.apache.curator:curator-client:jar:2.12.0:compile
[INFO] |  +- org.apache.curator:curator-recipes:jar:2.12.0:compile
[INFO] |  +- com.google.code.findbugs:jsr305:jar:3.0.0:compile
[INFO] |  +- org.apache.htrace:htrace-core4:jar:4.1.0-incubating:compile
[INFO] |  +- org.apache.zookeeper:zookeeper:jar:3.4.9:compile
[INFO] |  +- org.apache.commons:commons-compress:jar:1.4.1:compile
[INFO] |  |  \- org.tukaani:xz:jar:1.0:compile
[INFO] |  +- org.apache.kerby:kerb-simplekdc:jar:1.0.0:compile
[INFO] |  |  +- org.apache.kerby:kerb-client:jar:1.0.0:compile
[INFO] |  |  |  +- org.apache.kerby:kerby-config:jar:1.0.0:compile
[INFO] |  |  |  +- org.apache.kerby:kerb-core:jar:1.0.0:compile
[INFO] |  |  |  |  \- org.apache.kerby:kerby-pkix:jar:1.0.0:compile
[INFO] |  |  |  |     +- org.apache.kerby:kerby-asn1:jar:1.0.0:compile
[INFO] |  |  |  |     \- org.apache.kerby:kerby-util:jar:1.0.0:compile
[INFO] |  |  |  +- org.apache.kerby:kerb-common:jar:1.0.0:compile
[INFO] |  |  |  |  \- org.apache.kerby:kerb-crypto:jar:1.0.0:compile
[INFO] |  |  |  \- org.apache.kerby:kerb-util:jar:1.0.0:compile
[INFO] |  |  \- org.apache.kerby:kerb-admin:jar:1.0.0:compile
[INFO] |  |     +- org.apache.kerby:kerb-server:jar:1.0.0:compile
[INFO] |  |     |  \- org.apache.kerby:kerb-identity:jar:1.0.0:compile
[INFO] |  |     \- org.apache.kerby:kerby-xdr:jar:1.0.0:compile
[INFO] |  +- com.fasterxml.jackson.core:jackson-databind:jar:2.7.8:compile
[INFO] |  |  \- com.fasterxml.jackson.core:jackson-core:jar:2.7.8:compile
[INFO] |  +- org.codehaus.woodstox:stax2-api:jar:3.1.4:compile
[INFO] |  \- com.fasterxml.woodstox:woodstox-core:jar:5.0.3:compile
[INFO] +- org.apache.hadoop:hadoop-hdfs:test-jar:tests:3.0.0-beta1-SNAPSHOT:compile
[INFO] |  +- org.eclipse.jetty:jetty-util-ajax:jar:9.3.11.v20160721:compile
[INFO] |  +- commons-daemon:commons-daemon:jar:1.0.13:compile
[INFO] |  +- io.netty:netty:jar:3.10.5.Final:compile
[INFO] |  +- io.netty:netty-all:jar:4.0.23.Final:compile
[INFO] |  +- xerces:xercesImpl:jar:2.9.1:compile
[INFO] |  |  \- xml-apis:xml-apis:jar:1.3.04:compile

> Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, dom, etc.
> --------------------------------------------------------------------------------------
>
>                 Key: HADOOP-14672
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14672
>             Project: Hadoop Common
>          Issue Type: Sub-task
>            Reporter: Junping Du
>            Assignee: Bharat Viswanadham
>            Priority: Blocker
>         Attachments: HADOOP-14672.02.patch, HADOOP-14672.patch
>
>
> The shaded hadoop-client-minicluster shouldn't include any unshaded dependencies, but
we can see: javax, dom, sax, etc. are all unshaded.
> CC [~busbey]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message