hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6200) Create a separate jar for hdfs-client
Date Tue, 03 Mar 2015 19:19:06 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14345554#comment-14345554
] 

Vinod Kumar Vavilapalli commented on HDFS-6200:
-----------------------------------------------

+1000 for this proposal! (Not looked at the patch)
Reproducing my comments at HADOOP-11656.
 - Having a separate hdfs client JAR would vastly reduce the amount of classpath conflicts.
We have seen that in practice when we moved from Hadoop-1 MR to YARN having a leaner client
JAR avoided a whole lot of problems we had before even if wasn't perfectly done.
 - A lean client JAR is also a major help in how we rationalize stack wide rolling upgrades
- today NameNode is on the classpath of ResourceManager and RegionServer even if it doesn't
get used and so it is very hard to layout and upgrade bits easily.

> Create a separate jar for hdfs-client
> -------------------------------------
>
>                 Key: HDFS-6200
>                 URL: https://issues.apache.org/jira/browse/HDFS-6200
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Haohui Mai
>            Assignee: Haohui Mai
>         Attachments: HDFS-6200.000.patch, HDFS-6200.001.patch, HDFS-6200.002.patch, HDFS-6200.003.patch,
HDFS-6200.004.patch, HDFS-6200.005.patch, HDFS-6200.006.patch, HDFS-6200.007.patch
>
>
> Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs client. As discussed
in the hdfs-dev mailing list (http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser),
downstream projects are forced to bring in additional dependency in order to access hdfs.
The additional dependency sometimes can be difficult to manage for projects like Apache Falcon
and Apache Oozie.
> This jira proposes to create a new project, hadoop-hdfs-cliient, which contains the client
side of the hdfs code. Downstream projects can use this jar instead of the hadoop-hdfs to
avoid unnecessary dependency.
> Note that it does not break the compatibility of downstream projects. This is because
old downstream projects implicitly depend on hadoop-hdfs-client through the hadoop-hdfs jar.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message