hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wei-Chiu Chuang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10975) fsck -list-corruptfileblocks does not report corrupt EC files
Date Tue, 18 Oct 2016 16:52:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15585972#comment-15585972
] 

Wei-Chiu Chuang commented on HDFS-10975:
----------------------------------------

Hi [~tasanuma0829] thanks for making the patch available. I think your patch fixed most of
the problems, but there still appears to be one minor issue.

If I look at the output of testFsckMissingECFile, which intentionally removes 4 out of 9 datanodes:

{noformat}
Erasure Coded Block Groups:
 Total size:	393216 B
 Total files:	1
 Total block groups (validated):	1 (avg. block group size 393216 B)
  ********************************
  UNRECOVERABLE BLOCK GROUPS:	1 (100.0 %)
  CORRUPT FILES:	1
  MISSING BLOCK GROUPS:	1
  MISSING SIZE:		393216 B
  ********************************
 Minimally erasure-coded block groups:	0 (0.0 %)
 Over-erasure-coded block groups:	0 (0.0 %)
 Under-erasure-coded block groups:	0 (0.0 %)
 Unsatisfactory placement block groups:	0 (0.0 %)
 Default ecPolicy:		RS-DEFAULT-6-3-64k
 Average block group size:	5.0
 Missing block groups:		1
 Corrupt block groups:		0
 Missing internal blocks:	0 (0.0 %)
FSCK ended at Tue Oct 18 09:43:23 PDT 2016 in 2 milliseconds


The filesystem under path '/' is CORRUPT
{noformat}

The output says missing 0 internal blocks. Shouldn't it say missing 4 internal blocks?

> fsck -list-corruptfileblocks does not report corrupt EC files
> -------------------------------------------------------------
>
>                 Key: HDFS-10975
>                 URL: https://issues.apache.org/jira/browse/HDFS-10975
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>    Affects Versions: 3.0.0-alpha2
>            Reporter: Wei-Chiu Chuang
>            Assignee: Takanobu Asanuma
>         Attachments: HDFS-10975.1.patch
>
>
> HDFS-10826 fix fsck for corrupt EC files if no parameters are specified.
> However, if I change the test case added in HDFS-10826 (TestFsck#testFsckCorruptECFile)
to run "fsck -list-corruptfileblocks", the same test test failed because fsck reports no corrupt
files. 
> Interestingly, if I run "fsck -files -blocks -replicaDetails" then the test passed and
shows the corrupt file.
> Need to fix the discrepancy.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message