hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benoy Antony (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-7467) Provide storage tier information for a directory via fsck
Date Tue, 06 Jan 2015 22:23:35 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14266871#comment-14266871
] 

Benoy Antony edited comment on HDFS-7467 at 1/6/15 10:22 PM:
-------------------------------------------------------------

1. 
{quote}
Are all storage policies in fallback storage equivalent to other storage policies that this
output can always be fully described by the percentages that Tsz has suggested?
{quote}

There is a possibility that some storage tier combination may not belong to a storage policy.

My recommendation is to display the policy along with the combination if possible. If not,
display the combination. Lowercase for policy name is intentional.
{code}
Storage Policy                   # of blocks       % of blocks
cold(DISK:1,ARCHIVE:2)      340730          97.7393%
frozen(ARCHIVE:3)                   3928            1.1268%
DISK:2,ARCHIVE:2                   3122            0.8956%
warm(DISK:2,ARCHIVE:1)           748            0.2146%
DISK:1,ARCHIVE:3                        44            0.0126%
DISK:3,ARCHIVE:2                        30            0.0086%
DISK:3,ARCHIVE:1                          9            0.0026%
{code}

2.
{quote}
There should also be some warning messages as well in fsck for all files that are unable to
meet the requested ideal for their storage policy and are using fallback storage, perhaps
with a switch since that could become overly volumous output.
{quote}

This is a nice feature. Will look into that .


was (Author: benoyantony):
1. 
{quote}
Are all storage policies in fallback storage equivalent to other storage policies that this
output can always be fully described by the percentages that Tsz has suggested?
{quote}

There is a possibility that some storage tier combination may not belong to a storage policy.

My recommendation is to display the policy along with the combination if possible. If not,
display the combination. Lowercase for policy name is intentional.
{code}
Storage Policy                                  # of blocks       % of blocks
cold(DISK:1,ARCHIVE:2)                     340730          97.7393%
frozen(ARCHIVE:3)                                  3928            1.1268%
DISK:2,ARCHIVE:2                                  3122            0.8956%
warm(DISK:2,ARCHIVE:1)                         748            0.2146%
DISK:1,ARCHIVE:3                                      44            0.0126%
DISK:3,ARCHIVE:2                                      30            0.0086%
DISK:3,ARCHIVE:1                                        9            0.0026%         
{code}

2.
{quote}
There should also be some warning messages as well in fsck for all files that are unable to
meet the requested ideal for their storage policy and are using fallback storage, perhaps
with a switch since that could become overly volumous output.
{quote}

This is a nice feature. Will look into that .

> Provide storage tier information for a directory via fsck
> ---------------------------------------------------------
>
>                 Key: HDFS-7467
>                 URL: https://issues.apache.org/jira/browse/HDFS-7467
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: balancer & mover
>    Affects Versions: 2.6.0
>            Reporter: Benoy Antony
>            Assignee: Benoy Antony
>         Attachments: HDFS-7467.patch
>
>
> Currently _fsck_  provides information regarding blocks for a directory.
> It should be augmented to provide storage tier information (optionally). 
> The sample report could be as follows :
> {code}
> Storage Tier Combination        # of blocks       % of blocks
> DISK:1,ARCHIVE:2                  340730           97.7393%
>  
> ARCHIVE:3                           3928            1.1268%
>  
> DISK:2,ARCHIVE:2                    3122            0.8956%
>  
> DISK:2,ARCHIVE:1                     748            0.2146%
>  
> DISK:1,ARCHIVE:3                      44            0.0126%
>  
> DISK:3,ARCHIVE:2                      30            0.0086%
>  
> DISK:3,ARCHIVE:1                       9            0.0026%
> {code}
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message