hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11058) Implement 'hadoop fs -df' command for ViewFileSystem
Date Wed, 23 Nov 2016 03:29:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15688757#comment-15688757
] 

Hadoop QA commented on HDFS-11058:
----------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 11s{color} | {color:blue}
Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  0s{color} |
{color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m  0s{color}
| {color:green} The patch appears to include 3 new or modified test files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  2m 22s{color} | {color:blue}
Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 23s{color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m 13s{color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 39s{color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  2m 13s{color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 39s{color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 23s{color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 39s{color} |
{color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 15s{color} | {color:blue}
Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 23s{color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  9m 42s{color} |
{color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  9m 42s{color} | {color:red}
root generated 3 new + 711 unchanged - 3 fixed = 714 total (was 714) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 33s{color}
| {color:green} root: The patch generated 0 new + 143 unchanged - 4 fixed = 143 total (was
147) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  2m 12s{color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 35s{color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m  0s{color}
| {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m  3s{color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 43s{color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  7m 56s{color} | {color:green}
hadoop-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 69m 42s{color} | {color:red}
hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 33s{color}
| {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}130m 27s{color} | {color:black}
{color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestEncryptionZones |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:a9ad5d6 |
| JIRA Issue | HDFS-11058 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12840168/HDFS-11058.04.patch
|
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  unit  findbugs
 checkstyle  |
| uname | Linux 05dd024b5601 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 2bf9a15 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| javac | https://builds.apache.org/job/PreCommit-HDFS-Build/17642/artifact/patchprocess/diff-compile-javac-root.txt
|
| unit | https://builds.apache.org/job/PreCommit-HDFS-Build/17642/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
|  Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/17642/testReport/ |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/17642/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> Implement 'hadoop fs -df' command for ViewFileSystem   
> -------------------------------------------------------
>
>                 Key: HDFS-11058
>                 URL: https://issues.apache.org/jira/browse/HDFS-11058
>             Project: Hadoop HDFS
>          Issue Type: Task
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Manoj Govindassamy
>            Assignee: Manoj Govindassamy
>              Labels: viewfs
>         Attachments: HDFS-11058.01.patch, HDFS-11058.02.patch, HDFS-11058.03.patch, HDFS-11058.04.patch
>
>
> Df command doesn't seem to work well with ViewFileSystem. It always reports used data
as 0. Here is the client mount table configuration I am using against a federated clusters
of 2 NameNodes and 2 DataNoes. 
> {code}
>   1 <?xml version="1.0" ?>
>   2 <configuration>
>   3   <property>
>   4     <name>fs.defaultFS</name>
>   5     <value>viewfs://ClusterX/</value>
>   6   </property>
>   ..
>  11   <property>
>  12     <name>fs.default.name</name>
>  13     <value>viewfs://ClusterX/</value>
>  14   </property>
>  ..
>  23   <property>
>  24     <name>fs.viewfs.mounttable.ClusterX.link./nn0</name>
>  25     <value>hdfs://127.0.0.1:50001/</value>
>  26   </property>
>  27   <property>
>  28     <name>fs.viewfs.mounttable.ClusterX.link./nn1</name>
>  29     <value>hdfs://127.0.0.1:51001/</value>
>  30   </property>
>  31   <property>
>  32     <name>fs.viewfs.mounttable.ClusterX.link./nn2</name>
>  33     <value>hdfs://127.0.0.1:52001/nn2</value>
>  34   </property>
>  35   <property>
>  36     <name>fs.viewfs.mounttable.ClusterX.link./nn3</name>
>  37     <value>hdfs://127.0.0.1:52001/nn3</value>
>  38   </property>
>  39   <property>
>  40     <name>fs.viewfs.mounttable.ClusterY.linkMergeSlash</name>
>  41     <value>hdfs://127.0.0.1:50001/</value>
>  42   </property>
>  43 </configuration>
> {code}
> {{Df}} command always reports Size/Available as 8.0E and the usage as 0 for any federated
cluster. 
> {noformat}
> # hadoop fs -fs viewfs://ClusterX/ -df  /
> Filesystem                         Size  Used            Available  Use%
> viewfs://ClusterX/  9223372036854775807     0  9223372036854775807    0%
> # hadoop fs -fs viewfs://ClusterX/ -df  -h /
> Filesystem           Size  Used  Available  Use%
> viewfs://ClusterX/  8.0 E     0      8.0 E    0%
> # hadoop fs -fs viewfs://ClusterY/ -df  -h /
> Filesystem           Size  Used  Available  Use%
> viewfs://ClusterY/  8.0 E     0      8.0 E    0%
> {noformat}
> Whereas {{Du}} command seems to work as expected even with ViewFileSystem.
> {noformat}
> # hadoop fs -fs viewfs://ClusterY/ -du -h /
> 10.6 K  31.8 K  /build.log.16y
> 0       0       /user
> # hadoop fs -fs viewfs://ClusterX/ -du -h /
> 10.6 K  31.8 K  /nn0
> 0       0       /nn1
> 20.2 K  35.8 K  /nn3
> 40.6 K  34.3 K  /nn4
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message