hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manoj Govindassamy (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-11058) Implement 'hadoop fs -df' command for ViewFileSystem
Date Wed, 23 Nov 2016 00:29:58 GMT

     [ https://issues.apache.org/jira/browse/HDFS-11058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Manoj Govindassamy updated HDFS-11058:
--------------------------------------
    Attachment: HDFS-11058.04.patch

Thanks for the review [~andrew.wang]. Attaching v04 patch after the rebase to latest trunk.


> Implement 'hadoop fs -df' command for ViewFileSystem   
> -------------------------------------------------------
>
>                 Key: HDFS-11058
>                 URL: https://issues.apache.org/jira/browse/HDFS-11058
>             Project: Hadoop HDFS
>          Issue Type: Task
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Manoj Govindassamy
>            Assignee: Manoj Govindassamy
>              Labels: viewfs
>         Attachments: HDFS-11058.01.patch, HDFS-11058.02.patch, HDFS-11058.03.patch, HDFS-11058.04.patch
>
>
> Df command doesn't seem to work well with ViewFileSystem. It always reports used data
as 0. Here is the client mount table configuration I am using against a federated clusters
of 2 NameNodes and 2 DataNoes. 
> {code}
>   1 <?xml version="1.0" ?>
>   2 <configuration>
>   3   <property>
>   4     <name>fs.defaultFS</name>
>   5     <value>viewfs://ClusterX/</value>
>   6   </property>
>   ..
>  11   <property>
>  12     <name>fs.default.name</name>
>  13     <value>viewfs://ClusterX/</value>
>  14   </property>
>  ..
>  23   <property>
>  24     <name>fs.viewfs.mounttable.ClusterX.link./nn0</name>
>  25     <value>hdfs://127.0.0.1:50001/</value>
>  26   </property>
>  27   <property>
>  28     <name>fs.viewfs.mounttable.ClusterX.link./nn1</name>
>  29     <value>hdfs://127.0.0.1:51001/</value>
>  30   </property>
>  31   <property>
>  32     <name>fs.viewfs.mounttable.ClusterX.link./nn2</name>
>  33     <value>hdfs://127.0.0.1:52001/nn2</value>
>  34   </property>
>  35   <property>
>  36     <name>fs.viewfs.mounttable.ClusterX.link./nn3</name>
>  37     <value>hdfs://127.0.0.1:52001/nn3</value>
>  38   </property>
>  39   <property>
>  40     <name>fs.viewfs.mounttable.ClusterY.linkMergeSlash</name>
>  41     <value>hdfs://127.0.0.1:50001/</value>
>  42   </property>
>  43 </configuration>
> {code}
> {{Df}} command always reports Size/Available as 8.0E and the usage as 0 for any federated
cluster. 
> {noformat}
> # hadoop fs -fs viewfs://ClusterX/ -df  /
> Filesystem                         Size  Used            Available  Use%
> viewfs://ClusterX/  9223372036854775807     0  9223372036854775807    0%
> # hadoop fs -fs viewfs://ClusterX/ -df  -h /
> Filesystem           Size  Used  Available  Use%
> viewfs://ClusterX/  8.0 E     0      8.0 E    0%
> # hadoop fs -fs viewfs://ClusterY/ -df  -h /
> Filesystem           Size  Used  Available  Use%
> viewfs://ClusterY/  8.0 E     0      8.0 E    0%
> {noformat}
> Whereas {{Du}} command seems to work as expected even with ViewFileSystem.
> {noformat}
> # hadoop fs -fs viewfs://ClusterY/ -du -h /
> 10.6 K  31.8 K  /build.log.16y
> 0       0       /user
> # hadoop fs -fs viewfs://ClusterX/ -du -h /
> 10.6 K  31.8 K  /nn0
> 0       0       /nn1
> 20.2 K  35.8 K  /nn3
> 40.6 K  34.3 K  /nn4
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message