hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brandon Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4750) Support NFSv3 interface to HDFS
Date Thu, 25 Apr 2013 06:00:19 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641470#comment-13641470

Brandon Li commented on HDFS-4750:

{quote}This precludes having multiple NFS gateways in operation simultaneously for increased
throughput, right?{quote}
Not necessarily, it depends on the workloads and the application requirement.

Even for a regular NFS server mounted to multiple clients, it could have the same issue. One
way to synchronize the clienB-read-after-clienA-write is to use NFS lock manager(NLM) protocol(along
with Network Status Monitor (NSM) protocol). In the first phase, it seems a bit overkill for
the user cases we want to support.

Even in a data loading situation, I'd expect a set of several "gateway nodes" to be used in
round-robin in order to increase ingest throughput beyond what a single host can handle. 

Here what I want to mention is, as also in the proposal, one benefit of NFS support is to
make it easier to integrate HDFS into client's file system namespace. The performance of NFS
gateway is usually slower than using DFSClient directly. 

Loading file through NFS gateway can be faster than DFSClient only in a few cases, such as
unstable writes with no commit after them immediately. 

With that said, its performance can be improved in the future by a few ways, such as better
caching, pNFS support and etc.
> Support NFSv3 interface to HDFS
> -------------------------------
>                 Key: HDFS-4750
>                 URL: https://issues.apache.org/jira/browse/HDFS-4750
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>    Affects Versions: 3.0.0
>            Reporter: Brandon Li
>            Assignee: Brandon Li
>         Attachments: HADOOP-NFS-Proposal.pdf
> Access HDFS is usually done through HDFS Client or webHDFS. Lack of seamless integration
with client’s file system makes it difficult for users and impossible for some applications
to access HDFS. NFS interface support is one way for HDFS to have such easy integration.
> This JIRA is to track the NFS protocol support for accessing HDFS. With HDFS client,
webHDFS and the NFS interface, HDFS will be easier to access and be able support more applications
and use cases. 
> We will upload the design document and the initial implementation. 

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message