hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Milind Bhandarkar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5194) Robust support for alternate FsDatasetSpi implementations
Date Wed, 05 Feb 2014 22:36:12 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13892677#comment-13892677
] 

Milind Bhandarkar commented on HDFS-5194:
-----------------------------------------

This is an excellent proposal, and the right direction for HDFS. Over the last couple of years,
interest in Hadoop ecosystem has brought many storage system vendors to decide among the various
options, as to how they can integrate better with Hadoop. I have personally participated in
two such projects, which have taken two completely different approaches: 1. HDFS Wire-protocol
compatibility (where the external distributed file system appears to be HDFS) and 2. HDFS
API compatibility, which implements the FileSystem API.

This proposal takes the third route, which in my opinion is a better route, since it reduces
the overhead of keeping external storage systems integrated with HDFS, in spite of rapidly
evolving protocols / APIs, by reusing most of the plumbing of HDFS.

+1 on this Jira.

> Robust support for alternate FsDatasetSpi implementations
> ---------------------------------------------------------
>
>                 Key: HDFS-5194
>                 URL: https://issues.apache.org/jira/browse/HDFS-5194
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode, hdfs-client
>            Reporter: David Powell
>            Priority: Minor
>         Attachments: HDFS-5194.design.01222014.pdf, HDFS-5194.design.09112013.pdf, HDFS-5194.patch.09112013
>
>
> The existing FsDatasetSpi interface is well-positioned to permit extending Hadoop to
run natively on non-traditional storage architectures.  Before this can be done, however,
a number of gaps need to be addressed.  This JIRA documents those gaps, suggests some solutions,
and puts forth a sample implementation of some of the key changes needed.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message