hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mukul Kumar Singh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11968) ViewFS: StoragePolicies commands fail with HDFS federation
Date Mon, 02 Oct 2017 08:47:01 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16187767#comment-16187767

Mukul Kumar Singh commented on HDFS-11968:

The support for StoragePolicy was added to ViewFileSystem with HADOOP-13272.

The current patch makes use of this api to find out the underlying storage policies. Here
is the output for 2 namenode setup
[hdfs@y126 ~]$ /opt/hadoop/hadoop-3.1.0-SNAPSHOT/bin/hdfs storagepolicies -listPolicies
17/10/02 08:39:43 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your
platform... using builtin-java classes where applicable
Block Storage Policies:
	BlockStoragePolicy{COLD:2, storageTypes=[ARCHIVE], creationFallbacks=[], replicationFallbacks=[]}
	BlockStoragePolicy{WARM:5, storageTypes=[DISK, ARCHIVE], creationFallbacks=[DISK, ARCHIVE],
replicationFallbacks=[DISK, ARCHIVE]}
	BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}
	BlockStoragePolicy{ONE_SSD:10, storageTypes=[SSD, DISK], creationFallbacks=[SSD, DISK], replicationFallbacks=[SSD,
	BlockStoragePolicy{ALL_SSD:12, storageTypes=[SSD], creationFallbacks=[DISK], replicationFallbacks=[DISK]}
	BlockStoragePolicy{LAZY_PERSIST:15, storageTypes=[RAM_DISK, DISK], creationFallbacks=[DISK],

> ViewFS: StoragePolicies commands fail with HDFS federation
> ----------------------------------------------------------
>                 Key: HDFS-11968
>                 URL: https://issues.apache.org/jira/browse/HDFS-11968
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs
>    Affects Versions: 2.7.1
>            Reporter: Mukul Kumar Singh
>            Assignee: Mukul Kumar Singh
>         Attachments: HDFS-11968.001.patch, HDFS-11968.002.patch, HDFS-11968.003.patch,
HDFS-11968.004.patch, HDFS-11968.005.patch, HDFS-11968.006.patch, HDFS-11968.007.patch, HDFS-11968.008.patch,
HDFS-11968.009.patch, HDFS-11968.010.patch
> hdfs storagepolicies command fails with HDFS federation.
> For storage policies commands, a given user path should be resolved to a HDFS path and
> storage policy command should be applied onto the resolved HDFS path.
> {code}
>   static DistributedFileSystem getDFS(Configuration conf)
>       throws IOException {
>     FileSystem fs = FileSystem.get(conf);
>     if (!(fs instanceof DistributedFileSystem)) {
>       throw new IllegalArgumentException("FileSystem " + fs.getUri() +
>           " is not an HDFS file system");
>     }
>     return (DistributedFileSystem)fs;
>   }
> {code}

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message