hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hari Sekhon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7467) Provide storage tier information for a directory via fsck
Date Wed, 24 Dec 2014 10:57:14 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14258172#comment-14258172
] 

Hari Sekhon commented on HDFS-7467:
-----------------------------------

There does need to be a way to figure out if a given file or directory of files are using
fallback storage.

There should also be a global method of seeing if any files are using fallback storage as
an indicator that there isn't enough SSD for example.

Adding this information to fsck seems like a sensible way to go - the main question is how
to represent that information concisely.

Are all storage policies in fallback storage equivalent to other storage policies that this
output can always be fully described by the percentages that Tsz has suggested?

There should also be some warning messages as well in fsck for all files that are unable to
meet the requested ideal for their storage policy and are using fallback storage, perhaps
with a switch since that could become overly volumous output.

Regards,

Hari Sekhon
http://www.linkedin.com/in/harisekhon

> Provide storage tier information for a directory via fsck
> ---------------------------------------------------------
>
>                 Key: HDFS-7467
>                 URL: https://issues.apache.org/jira/browse/HDFS-7467
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: balancer & mover
>    Affects Versions: 2.6.0
>            Reporter: Benoy Antony
>            Assignee: Benoy Antony
>         Attachments: HDFS-7467.patch
>
>
> Currently _fsck_  provides information regarding blocks for a directory.
> It should be augmented to provide storage tier information (optionally). 
> The sample report could be as follows :
> {code}
> Storage Tier Combination        # of blocks       % of blocks
> DISK:1,ARCHIVE:2                  340730           97.7393%
>  
> ARCHIVE:3                           3928            1.1268%
>  
> DISK:2,ARCHIVE:2                    3122            0.8956%
>  
> DISK:2,ARCHIVE:1                     748            0.2146%
>  
> DISK:1,ARCHIVE:3                      44            0.0126%
>  
> DISK:3,ARCHIVE:2                      30            0.0086%
>  
> DISK:3,ARCHIVE:1                       9            0.0026%
> {code}
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message