hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jitendra Nath Pandey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11575) Supporting HDFS NFS gateway with Federated HDFS
Date Tue, 13 Jun 2017 04:49:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16047415#comment-16047415
] 

Jitendra Nath Pandey commented on HDFS-11575:
---------------------------------------------

A few comments:
   - Can we avoid calling fs.resolvePath(..)? This makes a server call and needs HDFS to be
up when NFS is being deployed. Not too bad, but decoupling would be better, as it has been
without this patch. One possibility is to make path qualified using file system uri. If creating
a URI is lightweight, we may not need to store path to URI mapping. The mount function already
makes a call to Namenode, URI creation could also be moved there.
   - To construct DFSClient, we could use this constructor {{DFSClient(URI nameNodeUri, Configuration
conf)}}, will save a step. The constructor should take care of HA case.
   - I am wondering why do we need 'hostMapAddress', since we are caching DFSClient itself,
the lookups to this map would be rare, in that case a DNS resolution is not that bad.
   -  {{ FileSystem[] childFileSystems = fs.getChildFileSystems()}} This line will get the
children file systems of the target as well. It might be better to get the target file systems
from the mount points. We may not need it altogether if hostMapAddress is removed.  
   - TestViewfsWithNfs3: Please add a test for rename to check for unsupported error. 
   - Add a test to start NFS service with viewfs over a non-hdfs file system. It is ok to
add it in a follow up jira.

> Supporting HDFS NFS gateway with Federated HDFS
> -----------------------------------------------
>
>                 Key: HDFS-11575
>                 URL: https://issues.apache.org/jira/browse/HDFS-11575
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: nfs
>            Reporter: Mukul Kumar Singh
>            Assignee: Mukul Kumar Singh
>         Attachments: HDFS-11575.001.patch, HDFS-11575.002.patch, HDFS-11575.003.patch,
HDFS-11575.004.patch, HDFS-11575.005.patch, SupportingNFSwithFederatedHDFS.pdf
>
>
> Currently HDFS NFS gateway only supports HDFS as the underlying filesystem.
> Federated HDFS with ViewFS helps in improving the scalability of the name nodes. However
NFS is not supported with ViewFS.
> With this change, ViewFS using HDFS as the underlying filesystem can be exported using
NFS. ViewFS mount table will be used to determine the exports which needs to be supported.
> Some important points
> 1) This patch only supports HDFS as the underlying filesystem for ViewFS.
> 2) This patch add support to add more than one export point in the NFS gateway
> 3) Root filesystem of the ViewFS will not be mountable for NFS gateway with ViewFS,
>    however this will not be the case for NFS gateway with HDFS
> 4) A filehandle now apart from the field will also contain an identifier to identify
the name node, this will be used to map to correct name node for file operations.
> Please find the attached pdf document which helps in explaining the design and the solution.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message