hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Koji Noguchi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6857) FsShell should report raw disk usage including replication factor
Date Tue, 14 Sep 2010 22:59:34 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12909502#action_12909502

Koji Noguchi commented on HADOOP-6857:

A little confused.  I thought "fs -count" shows hdfs usage same as "fs -du" at the thrid column.

[knoguchi ~]$ hadoop dfs -dus /user/knoguchi
hdfs://abc-nn1.com/user/knoguchi       2603203340273
[knoguchi ~]$ hadoop dfs -count /user/knoguchi
        1580        20624      2603203340273 hdfs://abc-nn1.com/user/knoguchi
[knoguchi ~]$ 
If quota is enabled on that dir and "-q" is passed, it would show the remaining raw space
[knoguchi ~]$ hadoop dfs -count -q /user/knoguchi
       50000           27796  13194139533312   5384528402193         1580        20624   
  2603203340273 hdfs://abc-nn1.com/user/knoguchi
[knoguchi ~]$ 
You can get the raw space usage then. (quota - raw\_remaining). 
However *this is only if you have quota enabled on that particular dir*.

> FsShell should report raw disk usage including replication factor
> -----------------------------------------------------------------
>                 Key: HADOOP-6857
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6857
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs
>            Reporter: Alex Kozlov
>             Fix For: 0.22.0
>         Attachments: show-space-consumed.txt
> Currently FsShell report HDFS usage with "hadoop fs -dus <path>" command.  Since
replication level is per file level, it would be nice to add raw disk usage including the
replication factor (maybe "hadoop fs -dus -raw <path>"?).  This will allow to assess
resource usage more accurately.  -- Alex K

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message