hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11058) Implement 'hadoop fs -df' command for ViewFileSystem
Date Wed, 26 Oct 2016 01:34:59 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15607035#comment-15607035
] 

Andrew Wang commented on HDFS-11058:
------------------------------------

Seems reasonable to me, though I'd prefer we put the new "Mounted on" as the last column and
left-aligned it (which is how it shows on my local df).

Adding this "Mounted on" column would be an incompatible change though. This is fine, but
do you intend to only show it when it's a ViewFileSystem?

> Implement 'hadoop fs -df' command for ViewFileSystem   
> -------------------------------------------------------
>
>                 Key: HDFS-11058
>                 URL: https://issues.apache.org/jira/browse/HDFS-11058
>             Project: Hadoop HDFS
>          Issue Type: Task
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Manoj Govindassamy
>            Assignee: Manoj Govindassamy
>
> Df command doesn't seem to work well with ViewFileSystem. It always reports used data
as 0. Here is the client mount table configuration I am using against a federated clusters
of 2 NameNodes and 2 DataNoes. 
> {code}
>   1 <?xml version="1.0" ?>
>   2 <configuration>
>   3   <property>
>   4     <name>fs.defaultFS</name>
>   5     <value>viewfs://ClusterX/</value>
>   6   </property>
>   ..
>  11   <property>
>  12     <name>fs.default.name</name>
>  13     <value>viewfs://ClusterX/</value>
>  14   </property>
>  ..
>  23   <property>
>  24     <name>fs.viewfs.mounttable.ClusterX.link./nn0</name>
>  25     <value>hdfs://127.0.0.1:50001/</value>
>  26   </property>
>  27   <property>
>  28     <name>fs.viewfs.mounttable.ClusterX.link./nn1</name>
>  29     <value>hdfs://127.0.0.1:51001/</value>
>  30   </property>
>  31   <property>
>  32     <name>fs.viewfs.mounttable.ClusterX.link./nn2</name>
>  33     <value>hdfs://127.0.0.1:52001/nn2</value>
>  34   </property>
>  35   <property>
>  36     <name>fs.viewfs.mounttable.ClusterX.link./nn3</name>
>  37     <value>hdfs://127.0.0.1:52001/nn3</value>
>  38   </property>
>  39   <property>
>  40     <name>fs.viewfs.mounttable.ClusterY.linkMergeSlash</name>
>  41     <value>hdfs://127.0.0.1:50001/</value>
>  42   </property>
>  43 </configuration>
> {code}
> {{Df}} command always reports Size/Available as 8.0E and the usage as 0 for any federated
cluster. 
> {noformat}
> # hadoop fs -fs viewfs://ClusterX/ -df  /
> Filesystem                         Size  Used            Available  Use%
> viewfs://ClusterX/  9223372036854775807     0  9223372036854775807    0%
> # hadoop fs -fs viewfs://ClusterX/ -df  -h /
> Filesystem           Size  Used  Available  Use%
> viewfs://ClusterX/  8.0 E     0      8.0 E    0%
> # hadoop fs -fs viewfs://ClusterY/ -df  -h /
> Filesystem           Size  Used  Available  Use%
> viewfs://ClusterY/  8.0 E     0      8.0 E    0%
> {noformat}
> Whereas {{Du}} command seems to work as expected even with ViewFileSystem.
> {noformat}
> # hadoop fs -fs viewfs://ClusterY/ -du -h /
> 10.6 K  31.8 K  /build.log.16y
> 0       0       /user
> # hadoop fs -fs viewfs://ClusterX/ -du -h /
> 10.6 K  31.8 K  /nn0
> 0       0       /nn1
> 20.2 K  35.8 K  /nn3
> 40.6 K  34.3 K  /nn4
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message