hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brandon Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6549) Add support for accessing the NFS gateway from the AIX NFS client
Date Wed, 18 Jun 2014 21:42:24 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14036443#comment-14036443

Brandon Li commented on HDFS-6549:

This sometimes happens with some traditional NFS server/client implementations: when the NFS
client does READDIR/READDIRPLUS with a huge directory which keeps changing, the READDIR/READDIRPLUS
can never finish because the verifier changes and NFS client has to restart the listing request
before it finishes. But based on what you described, the AIX client is a different case.

Large directory may need multiple READDIR/READDIRPLUS requests. Many NFS client implementations
send 0 as the verifier for the first READDIR/READDIRPLUS RPC request. In the AIX case, after
it grabs a verifier from server it will always use the same verifier even in the initial READDIR/READIRPLUS
request when it accesses the same directory again. Is my understanding correct?

> Add support for accessing the NFS gateway from the AIX NFS client
> -----------------------------------------------------------------
>                 Key: HDFS-6549
>                 URL: https://issues.apache.org/jira/browse/HDFS-6549
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: nfs
>    Affects Versions: 2.4.0
>            Reporter: Aaron T. Myers
>            Assignee: Aaron T. Myers
>         Attachments: HDFS-6549.patch
> We've identified two issues when trying to access the HDFS NFS Gateway from an AIX NFS
> # In the case of COMMITs, the AIX NFS client will always send 4096, or a multiple of
the page size, for the offset to be committed, even if fewer bytes than this have ever, or
will ever, be written to the file. This will cause a write to a file from the AIX NFS client
to hang on close unless the size of that file is a multiple of 4096.
> # In the case of READDIR and READDIRPLUS, the AIX NFS client will send the same cookie
verifier for a given directory seemingly forever after that directory is first accessed over
NFS, instead of getting a new cookie verifier for every set of incremental readdir calls.
This means that if a directory's mtime ever changes, the FS must be unmounted/remounted before
readdir calls on that dir from AIX will ever succeed again.
> From my interpretation of RFC-1813, the NFS Gateway is in fact doing the correct thing
in both cases, but we can introduce simple changes on the NFS Gateway side to be able to optionally
work around these incompatibilities.

This message was sent by Atlassian JIRA

View raw message