hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haohui Mai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12499) Create a client jar for hadoop-common
Date Wed, 21 Oct 2015 20:08:27 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14967781#comment-14967781
] 

Haohui Mai commented on HADOOP-12499:
-------------------------------------

bq. what bits do you propose to cut here? Because the only bit you'd say is pure server is
the jetty stuff in org/apache/hadoop/http. Everything else: IPC, security, utils, service
model, they're all client-side as well. Yes, IPC server is arguably server-side, but it is
used in all hadoop IPC servers, rather than just the core services.

My proposal is that only classes that are used to connect YARN/HDFS should reside in {{hadoop-common-client}}.
That excludes RPCServer, HA servers, jmx, metrics, log, to just name a few.

bq. It seems to me, that a hadoop common client JAR is simply hadoop-common.jar with all the
server-side dependencies stripped from it's POM.

That might be close. The main motivation is to have clean dependency at compile time. For
example, {{hadoop-common-client}} should not depend on {{guava}}. While it's possible exclude
{{guava}} from the pom, it's practically impossible (1) to tell whether it works until the
deployment time, and (2) to enforce not bringing in these troublesome dependency in {{hadoop-common-client}}.



> Create a client jar for hadoop-common
> -------------------------------------
>
>                 Key: HADOOP-12499
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12499
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Haohui Mai
>            Assignee: Haohui Mai
>
> HDFS-6200 has refactor the HDFS client implementations into the {{hadoop-hdfs-client}}
module. However, the client module still depends on {{hadoop-common}} which contains both
the implementation of hadoop server and clients.
> This jira proposes to separate client-side implementation of {{hadoop-common}} to a new
module {{hadoop-common-client}} so that both the yarn and the hdfs client no longer need to
transitively bring in dependency in the server side.
> Per feedbacks from [~steve_l] and [~cmccabe], the development will happen in a separate
branch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message