hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Milind Bhandarkar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5324) Make Namespace implementation pluggable in the namenode
Date Mon, 14 Oct 2013 19:46:42 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13794398#comment-13794398

Milind Bhandarkar commented on HDFS-5324:


As I have replied to Sanjay, I am not proposing a stable API within HDFS for implementing
Namespaces, and similar to what you pointed out quoting from linux kernel mailing list, the
proposed API will evolve rapidly, and will change incompatibly.

I have participated in at least one project to implement a HDFS protocol-compatible external
file-system, and found that it was possible to do to make other proprietary (or open) file
systems "appear" as if they are HDFS (not merely Hadoop-compatible). However, it does not
allow the HDFS community itself to experiment with different namespace implementations, within

The primary audience for the pluggability proposal is, HDFS developers, who want to develop
more scalable implementations of namespace, without having to rewrite the entire namenode

I think Bobby  Evans made the point much better than I could, on the mailing list: All the
approaches for scaling namespace is based on tradeoffs. In order for evaluating the right
mix of complexity, scalability, and ease of use and deployment, we need to first allow experimentation,
and independent parallel development. That is what this pluggability proposal is aimed at

> Make Namespace implementation pluggable in the namenode
> -------------------------------------------------------
>                 Key: HDFS-5324
>                 URL: https://issues.apache.org/jira/browse/HDFS-5324
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>    Affects Versions: 2.1.1-beta
>         Environment: All
>            Reporter: Milind Bhandarkar
>            Assignee: Milind Bhandarkar
>             Fix For: 3.0.0
>         Attachments: AbstractNamesystem.java
> For the last couple of months, we have been working on making Namespace
> implementation in the namenode pluggable. We have demonstrated that it can
> be done without major surgery on the namenode, and does not have noticeable
> performance impact. We would like to contribute it back to Apache HDFS.

This message was sent by Atlassian JIRA

View raw message