hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12499) Create a client jar for hadoop-common
Date Wed, 21 Oct 2015 19:06:27 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14967676#comment-14967676

Steve Loughran commented on HADOOP-12499:

what bits do you propose to cut here? Because the only bit you'd say is pure server is the
jetty stuff in {{org/apache/hadoop/http}}. Everything else: IPC, security, utils, service
model, they're all client-side as well. Yes, IPC server is arguably server-side, but it is
used in all hadoop IPC servers, rather than just the core services.

It's very different here from say HDFS, which can have a hard split from client and the NN
& DN services, and YARN, which has already had a split.

It seems to me, that a hadoop common client JAR is simply hadoop-common.jar with all the server-side
dependencies stripped from it's POM.

> Create a client jar for hadoop-common
> -------------------------------------
>                 Key: HADOOP-12499
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12499
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Haohui Mai
>            Assignee: Haohui Mai
> HDFS-6200 has refactor the HDFS client implementations into the {{hadoop-hdfs-client}}
module. However, the client module still depends on {{hadoop-common}} which contains both
the implementation of hadoop server and clients.
> This jira proposes to separate client-side implementation of {{hadoop-common}} to a new
module {{hadoop-common-client}} so that both the yarn and the hdfs client no longer need to
transitively bring in dependency in the server side.
> Per feedbacks from [~steve_l] and [~cmccabe], the development will happen in a separate

This message was sent by Atlassian JIRA

View raw message