hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Busbey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14672) Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, dom, etc.
Date Fri, 28 Jul 2017 02:08:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16104325#comment-16104325

Sean Busbey commented on HADOOP-14672:

Ideally, it is true that these test jars are not marked as public or LimitedPrivate so shouldn't
be used by downstream projects. However, it would be great if we can check those main stream
projects, such as HBase, Hive, etc. won't use them at all. Otherwise, some related tests for
downstream projects could get break. I like the idea to separate it into a dedicated JIRA
for additional discussion and verification. Sean Busbey, what do you think?

I think it's broken that we include them and we should be using the new client-facing jars
in our new major version to push against prior broken behavior whenever we can. I'm fine with
doing this in a follow on, and will take my concerns over to HADOOP-14685.

Xerces sounds like a complicated issue across different JVMs. Can we just leave it there or
treat it as normal third party classes? Any side-effect if we shade Xerces classes as third
party classes?

According to my understanding shading Xerces classes, will not cause issue. It will work across
different JVM's with out any issue.

Xerces can be difficult, because it provides an alternate implementation for the basic XML
building blocks that come in the JVM. If we're including it relocated, we'll either a) do
it wrong and cause the JVM to break when folks try to use the built in xml related classes
or b) make diagnosing a problem super hard when we correctly load an alternate xml parsing

If we're actually using it, that's fine; let's relocate and bundle it. But let's be sure we're
actually using it. I didn't see anyone chase down if we need it beyond the fsimage xml tool
thing, which isn't even a part of what we're trying to provide with this artifact.

> Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, dom, etc.
> --------------------------------------------------------------------------------------
>                 Key: HADOOP-14672
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14672
>             Project: Hadoop Common
>          Issue Type: Sub-task
>            Reporter: Junping Du
>            Assignee: Bharat Viswanadham
>            Priority: Blocker
>         Attachments: HADOOP-14672.02.patch, HADOOP-14672.03.patch, HADOOP-14672.04.patch,
> The shaded hadoop-client-minicluster shouldn't include any unshaded dependencies, but
we can see: javax, dom, sax, etc. are all unshaded.
> CC [~busbey]

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message