hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Koji Noguchi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6857) FsShell should report raw disk usage including replication factor
Date Tue, 14 Sep 2010 22:59:34 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12909502#action_12909502
] 

Koji Noguchi commented on HADOOP-6857:
--------------------------------------

A little confused.  I thought "fs -count" shows hdfs usage same as "fs -du" at the thrid column.

{noformat}
[knoguchi ~]$ hadoop dfs -dus /user/knoguchi
hdfs://abc-nn1.com/user/knoguchi       2603203340273
[knoguchi ~]$ hadoop dfs -count /user/knoguchi
        1580        20624      2603203340273 hdfs://abc-nn1.com/user/knoguchi
[knoguchi ~]$ 
{noformat}
If quota is enabled on that dir and "-q" is passed, it would show the remaining raw space
available. 
{noformat}
[knoguchi ~]$ hadoop dfs -count -q /user/knoguchi
       50000           27796  13194139533312   5384528402193         1580        20624   
  2603203340273 hdfs://abc-nn1.com/user/knoguchi
[knoguchi ~]$ 
{noformat}
You can get the raw space usage then. (quota - raw\_remaining). 
However *this is only if you have quota enabled on that particular dir*.


> FsShell should report raw disk usage including replication factor
> -----------------------------------------------------------------
>
>                 Key: HADOOP-6857
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6857
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs
>            Reporter: Alex Kozlov
>             Fix For: 0.22.0
>
>         Attachments: show-space-consumed.txt
>
>
> Currently FsShell report HDFS usage with "hadoop fs -dus <path>" command.  Since
replication level is per file level, it would be nice to add raw disk usage including the
replication factor (maybe "hadoop fs -dus -raw <path>"?).  This will allow to assess
resource usage more accurately.  -- Alex K

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message