hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6857) FsShell should report raw disk usage including replication factor
Date Wed, 15 Sep 2010 16:22:34 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12909788#action_12909788

Eli Collins commented on HADOOP-6857:

I agree with Koji, seeing raw disk space usage should be easy to get from the CLI, you shouldn't
have to enable a quota on a directory to see it, eg see below that nothing indicates raw disk

~ $ hadoop fs -mkdir dir
~ $ hadoop fs -put f3mb dir
~ $ hadoop fs -dus dir
hdfs://haus01.sf.cloudera.com:10020/user/eli/dir	3145728
~ $ hadoop fs -count -q dir
        none             inf            none             inf            1            1   
        3145728 hdfs://haus01.sf.cloudera.com:10020/user/eli/dir

It also sounds like the original issue here was that disk space usage should not have been
added to count if it was already available in dus. Even if we added raw disk usage to "count",
it's wonky that a user can logical disk usage with "du" but has to switch over to a separate
"count" command to get raw disk usage. Why not make all disk usage available from a single

Sound reasonable Nicholas?

> FsShell should report raw disk usage including replication factor
> -----------------------------------------------------------------
>                 Key: HADOOP-6857
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6857
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs
>            Reporter: Alex Kozlov
>             Fix For: 0.22.0
>         Attachments: show-space-consumed.txt
> Currently FsShell report HDFS usage with "hadoop fs -dus <path>" command.  Since
replication level is per file level, it would be nice to add raw disk usage including the
replication factor (maybe "hadoop fs -dus -raw <path>"?).  This will allow to assess
resource usage more accurately.  -- Alex K

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message