hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mikhail Erofeev (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12292) Federation: Support viewfs:// schema path for DfsAdmin commands
Date Fri, 18 Aug 2017 19:48:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16133547#comment-16133547

Mikhail Erofeev commented on HDFS-12292:

Hi [~cliang], thanks for pointing out to this.  
Indeed, my scope was too broad, I moved a call to resolveFsPath() from Command to DFSAdmin.DFSAdminCommand.
I also added an optimisation check for the same fs in resolveFsPath(), though for now all
paths are normalized from expandAsGlob() and don't benefit from it.
As of you original question viewfs/non-viewfs, I fixed almost as you suggested, but I checked
for DistributedFileSystem instead of just ViewFileSystem, as other wrapper-fs (ChRootedFileSystem,
FilterFileSystem, HarFileSystem) are also can fail with this exception.
After these fixes TestHDFSCLI no longer needs changes as was in the previous version!
Thanks again.

> Federation: Support viewfs:// schema path for DfsAdmin commands
> ---------------------------------------------------------------
>                 Key: HDFS-12292
>                 URL: https://issues.apache.org/jira/browse/HDFS-12292
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: federation
>            Reporter: Mikhail Erofeev
>            Assignee: Mikhail Erofeev
>         Attachments: HDFS-12292-002.patch, HDFS-12292-003.patch, HDFS-12292-004.patch,
> Motivation:
> As of now, clients need to specify a nameservice when a cluster is federated, otherwise,
the exception is fired:
> {code}
> hdfs dfsadmin -setQuota 10 viewfs://vfs-root/user/uname
> setQuota: FileSystem viewfs://vfs-root/ is not an HDFS file system
> # with fs.defaultFS = viewfs://vfs-root/
> hdfs dfsadmin -setQuota 10 vfs-root/user/uname
> setQuota: FileSystem viewfs://vfs-root/ is not an HDFS file system
> # works fine thanks to https://issues.apache.org/jira/browse/HDFS-11432
> hdfs dfsadmin -setQuota 10 hdfs://users-fs/user/uname
> {code}
> This creates inconvenience, inability to rely on fs.defaultFS and forces to create client-side
mappings for management scripts
> Implementation:
> PathData that is passed to commands should be resolved to its actual FileSystem
> Result:
> ViewFS will be resolved to the actual HDFS file system

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message