hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manoj Govindassamy (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-11058) Implement 'hadoop fs -df' command for ViewFileSystem
Date Wed, 26 Oct 2016 00:04:58 GMT
Manoj Govindassamy created HDFS-11058:
-----------------------------------------

             Summary: Implement 'hadoop fs -df' command for ViewFileSystem   
                 Key: HDFS-11058
                 URL: https://issues.apache.org/jira/browse/HDFS-11058
             Project: Hadoop HDFS
          Issue Type: Task
    Affects Versions: 3.0.0-alpha1
            Reporter: Manoj Govindassamy
            Assignee: Manoj Govindassamy


Df command doesn't seem to work well with ViewFileSystem. It always reports used data as 0.
Here is the client mount table configuration I am using against a federated clusters of 2
NameNodes and 2 DataNoes. 

{code}
  1 <?xml version="1.0" ?>
  2 <configuration>
  3   <property>
  4     <name>fs.defaultFS</name>
  5     <value>viewfs://ClusterX/</value>
  6   </property>
  ..
 11   <property>
 12     <name>fs.default.name</name>
 13     <value>viewfs://ClusterX/</value>
 14   </property>
 ..
 23   <property>
 24     <name>fs.viewfs.mounttable.ClusterX.link./nn0</name>
 25     <value>hdfs://127.0.0.1:50001/</value>
 26   </property>
 27   <property>
 28     <name>fs.viewfs.mounttable.ClusterX.link./nn1</name>
 29     <value>hdfs://127.0.0.1:51001/</value>
 30   </property>
 31   <property>
 32     <name>fs.viewfs.mounttable.ClusterX.link./nn2</name>
 33     <value>hdfs://127.0.0.1:52001/nn2</value>
 34   </property>
 35   <property>
 36     <name>fs.viewfs.mounttable.ClusterX.link./nn3</name>
 37     <value>hdfs://127.0.0.1:52001/nn3</value>
 38   </property>
 39   <property>
 40     <name>fs.viewfs.mounttable.ClusterY.linkMergeSlash</name>
 41     <value>hdfs://127.0.0.1:50001/</value>
 42   </property>
 43 </configuration>
{code}


{{Df}} command always reports Size/Available as 8.0E and the usage as 0 for any federated
cluster. 

{noformat}
# hadoop fs -fs viewfs://ClusterX/ -df  /
Filesystem                         Size  Used            Available  Use%
viewfs://ClusterX/  9223372036854775807     0  9223372036854775807    0%

# hadoop fs -fs viewfs://ClusterX/ -df  -h /
Filesystem           Size  Used  Available  Use%
viewfs://ClusterX/  8.0 E     0      8.0 E    0%

# hadoop fs -fs viewfs://ClusterY/ -df  -h /
Filesystem           Size  Used  Available  Use%
viewfs://ClusterY/  8.0 E     0      8.0 E    0%
{noformat}

Whereas {{Du}} command seems to work as expected even with ViewFileSystem.

{noformat}
# hadoop fs -fs viewfs://ClusterY/ -du -h /
10.6 K  31.8 K  /build.log.16y
0       0       /user

# hadoop fs -fs viewfs://ClusterX/ -du -h /
10.6 K  31.8 K  /nn0
0       0       /nn1
20.2 K  35.8 K  /nn3
40.6 K  34.3 K  /nn4

{noformat}






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org


Mime
View raw message