hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mukul Kumar Singh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11968) ViewFS: StoragePolicies commands fail with HDFS federation
Date Tue, 19 Sep 2017 08:21:01 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16171294#comment-16171294
] 

Mukul Kumar Singh commented on HDFS-11968:
------------------------------------------

Thanks for the review [~surendrasingh],

1) ViewFileSystem#resolveStoragePolicyPath() why this method is required ?. fsState.resolve
already throwing FileNotFoundException. Only thing is proper exception message is missing
in fsState.resolve.
bq. Fixed by modifying the message in fsState.resolve.

2) What is realId in BlockStoragePolicySpi? 
bq. As you have already noted, this was added to get the local policy id. With ViewFilesystem
we cannot use the HdfsFileStatus.
Hence I thought of adding another field to fetch the local policy id of the command. This
also avoid two rpc calls one for
a) getting the filestatus and b) for getting the list of all the storage policies for a namenode.

I have updated the patch for 1) Please have a look and let me know of your comments.

> ViewFS: StoragePolicies commands fail with HDFS federation
> ----------------------------------------------------------
>
>                 Key: HDFS-11968
>                 URL: https://issues.apache.org/jira/browse/HDFS-11968
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs
>    Affects Versions: 2.7.1
>            Reporter: Mukul Kumar Singh
>            Assignee: Mukul Kumar Singh
>         Attachments: HDFS-11968.001.patch, HDFS-11968.002.patch, HDFS-11968.003.patch,
HDFS-11968.004.patch, HDFS-11968.005.patch, HDFS-11968.006.patch, HDFS-11968.007.patch
>
>
> hdfs storagepolicies command fails with HDFS federation.
> For storage policies commands, a given user path should be resolved to a HDFS path and
> storage policy command should be applied onto the resolved HDFS path.
> {code}
>   static DistributedFileSystem getDFS(Configuration conf)
>       throws IOException {
>     FileSystem fs = FileSystem.get(conf);
>     if (!(fs instanceof DistributedFileSystem)) {
>       throw new IllegalArgumentException("FileSystem " + fs.getUri() +
>           " is not an HDFS file system");
>     }
>     return (DistributedFileSystem)fs;
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message