hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brock Noland (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4750) Support NFSv3 interface to HDFS
Date Sat, 27 Apr 2013 17:16:18 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13643739#comment-13643739

Brock Noland commented on HDFS-4750:


Yes, this a major concern. Any clients will have to be well behaved and explicitly not perform
random writes on the client side. With NFS4, as long as the client application is not performing
random writes, the linux NFS4 client does not appear attempt any random writes. As I mentioned
earlier I thought I saw the linux NFS3 client perform random writes for a sequentially writing
client. Hopefully I was mistaken.

In regard to handling the normal write-reordering which occurs under both NFS3 and 4, the
approach I took in the NFS4 proxy was to buffer any non-sequential writes until I could write
them sequentially. As you said previously this can lead unbounded memory consumption. Therefore
in the NFS4 proxy, if a write request is greater than 1MB from the current file offset, it's
written out to a log file and the in-memory object simply stores the file name, offset, and
length. I've found this method to work quite well.

> Support NFSv3 interface to HDFS
> -------------------------------
>                 Key: HDFS-4750
>                 URL: https://issues.apache.org/jira/browse/HDFS-4750
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>    Affects Versions: 3.0.0
>            Reporter: Brandon Li
>            Assignee: Brandon Li
>         Attachments: HADOOP-NFS-Proposal.pdf
> Access HDFS is usually done through HDFS Client or webHDFS. Lack of seamless integration
with client’s file system makes it difficult for users and impossible for some applications
to access HDFS. NFS interface support is one way for HDFS to have such easy integration.
> This JIRA is to track the NFS protocol support for accessing HDFS. With HDFS client,
webHDFS and the NFS interface, HDFS will be easier to access and be able support more applications
and use cases. 
> We will upload the design document and the initial implementation. 

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message