hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Takanobu Asanuma (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-10826) The result of fsck should be CRITICAL when there are unrecoverable ec block groups.
Date Fri, 23 Sep 2016 04:41:20 GMT

     [ https://issues.apache.org/jira/browse/HDFS-10826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Takanobu Asanuma updated HDFS-10826:
------------------------------------
    Attachment: HDFS-10826.2.patch

I uploaded a new patch.

There are two types of problems, corrupted ec files and missing ec files. This patch fixes
BlockManager to handle corrupted ec files and NamenodeFsck to handle missing ec files. It
also includes the unit tests.

> The result of fsck should be CRITICAL when there are unrecoverable ec block groups.
> -----------------------------------------------------------------------------------
>
>                 Key: HDFS-10826
>                 URL: https://issues.apache.org/jira/browse/HDFS-10826
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: erasure-coding
>            Reporter: Takanobu Asanuma
>            Assignee: Takanobu Asanuma
>         Attachments: HDFS-10826.2.patch, HDFS-10826.WIP.1.patch
>
>
> For RS-6-3, when there is one ec block group and
> 1) 0~3 out of 9 internal blocks are missing, the result of fsck is HEALTY.
> 2) 4~8 out of 9 internal blocks are missing, the result of fsck is HEALTY.
> {noformat}
> Erasure Coded Block Groups:
>  Total size:    536870912 B
>  Total files:   1
>  Total block groups (validated):        1 (avg. block group size 536870912 B)
>   ********************************
>   UNRECOVERABLE BLOCK GROUPS:   1 (100.0 %)
>   ********************************
>  Minimally erasure-coded block groups:  0 (0.0 %)
>  Over-erasure-coded block groups:       0 (0.0 %)
>  Under-erasure-coded block groups:      1 (100.0 %)
>  Unsatisfactory placement block groups: 0 (0.0 %)
>  Default ecPolicy:              RS-DEFAULT-6-3-64k
>  Average block group size:      5.0
>  Missing block groups:          0
>  Corrupt block groups:          0
>  Missing internal blocks:       4 (44.444443 %)
> FSCK ended at Wed Aug 31 13:42:05 JST 2016 in 4 milliseconds
> The filesystem under path '/' is HEALTHY
> {noformat}
> 3) 9 out of 9 internal blocks are missing, the result of fsck is CRITICAL. (Because it
is regarded as a missing block group.)
> In case 2), the result should be CRITICAL since the ec block group is unrecoverable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message