hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Takanobu Asanuma (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7687) Change fsck to support EC files
Date Tue, 21 Apr 2015 13:06:59 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14504914#comment-14504914
] 

Takanobu Asanuma commented on HDFS-7687:
----------------------------------------

Sorry for my late work, [~szetszwo].
I'm mainly changing codes in {{NamenodeFsck.check}} to handle EC and I'm going to add some
metrics for EC, referring to replication. Please would you check these metrics?

{{Total EC block groups}}:
The number of all EC block groups on the HDFS.

{{Minimally stored block groups}}:
The number of EC block groups which have enough blocks to recover. For example, in (6,3)-Reed-Solomon,
these groups have 6 blocks at least.

{{Over EC block groups}}:
The number of EC block groups which have excess blocks for some reason. For example, in (6,3)-Reed-Solomon,
these groups have more than 9 blocks. (Are there these cases?)

{{Under EC block groups}}:
The number of EC block groups which have lost blocks.

{{Mis EC block groups}}:
The number of EC block groups whose rack locations are invalid.

{{Default EC schema}}:
This is usually "SYS-DEFAULT-RS-6-3". I think this will be set by a configuration file later.

{{Corrupt EC block groups}}:
The number of EC block groups which don't have enough blocks to recovery. For example, in
(6,3)-Reed-Solomon, these groups have less than 6 blocks, so they can't recover.

> Change fsck to support EC files
> -------------------------------
>
>                 Key: HDFS-7687
>                 URL: https://issues.apache.org/jira/browse/HDFS-7687
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: Takanobu Asanuma
>
> We need to change fsck so that it can detect "under replicated" and corrupted EC files.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message