hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Milind Bhandarkar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5324) Make Namespace implementation pluggable in the namenode
Date Mon, 14 Oct 2013 19:46:42 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13794398#comment-13794398
] 

Milind Bhandarkar commented on HDFS-5324:
-----------------------------------------

Colin,

As I have replied to Sanjay, I am not proposing a stable API within HDFS for implementing
Namespaces, and similar to what you pointed out quoting from linux kernel mailing list, the
proposed API will evolve rapidly, and will change incompatibly.

I have participated in at least one project to implement a HDFS protocol-compatible external
file-system, and found that it was possible to do to make other proprietary (or open) file
systems "appear" as if they are HDFS (not merely Hadoop-compatible). However, it does not
allow the HDFS community itself to experiment with different namespace implementations, within
HDFS.

The primary audience for the pluggability proposal is, HDFS developers, who want to develop
more scalable implementations of namespace, without having to rewrite the entire namenode
code.

I think Bobby  Evans made the point much better than I could, on the mailing list: All the
approaches for scaling namespace is based on tradeoffs. In order for evaluating the right
mix of complexity, scalability, and ease of use and deployment, we need to first allow experimentation,
and independent parallel development. That is what this pluggability proposal is aimed at
doing.

> Make Namespace implementation pluggable in the namenode
> -------------------------------------------------------
>
>                 Key: HDFS-5324
>                 URL: https://issues.apache.org/jira/browse/HDFS-5324
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>    Affects Versions: 2.1.1-beta
>         Environment: All
>            Reporter: Milind Bhandarkar
>            Assignee: Milind Bhandarkar
>             Fix For: 3.0.0
>
>         Attachments: AbstractNamesystem.java
>
>
> For the last couple of months, we have been working on making Namespace
> implementation in the namenode pluggable. We have demonstrated that it can
> be done without major surgery on the namenode, and does not have noticeable
> performance impact. We would like to contribute it back to Apache HDFS.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message