hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "J.Andreina (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-8533) Mismatch in displaying the "MissingBlock" count in fsck and in other metric reports
Date Thu, 04 Jun 2015 10:35:37 GMT

     [ https://issues.apache.org/jira/browse/HDFS-8533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

J.Andreina updated HDFS-8533:
-----------------------------
    Description: 
Number of DN = 2
Step 1: Write a file with replication factor - 3 .
Step 2: Corrupt a replica in DN1
Step 3: DN2 is down. 

Missing Block count in  report is as follows
Fsck report                                    : *0*
Jmx, "dfsadmin -report" , UI, logs : *1*

In fsck , only block whose replicas are all missed and not been corrupted are counted 
{code}
if (totalReplicasPerBlock == 0 && !isCorrupt) {
        // If the block is corrupted, it means all its available replicas are
        // corrupted. We don't mark it as missing given these available replicas
        // might still be accessible as the block might be incorrectly marked as
        // corrupted by client machines.
{code}

While in other reports even if all the replicas are corrupted , block is been considered as
missed.

Please provide your thoughts : can we make missing block count consistent across all the reports
same as implemented for fsck?



  was:
Number of DN = 2
Step 1: Write a file with replication factor - 3 .
Step 2: Corrupt a replica in DN1
Step 3: DN2 is down. 

Fsck report 



> Mismatch in displaying the "MissingBlock" count in fsck and in other metric reports
> -----------------------------------------------------------------------------------
>
>                 Key: HDFS-8533
>                 URL: https://issues.apache.org/jira/browse/HDFS-8533
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: J.Andreina
>            Assignee: J.Andreina
>            Priority: Critical
>
> Number of DN = 2
> Step 1: Write a file with replication factor - 3 .
> Step 2: Corrupt a replica in DN1
> Step 3: DN2 is down. 
> Missing Block count in  report is as follows
> Fsck report                                    : *0*
> Jmx, "dfsadmin -report" , UI, logs : *1*
> In fsck , only block whose replicas are all missed and not been corrupted are counted

> {code}
> if (totalReplicasPerBlock == 0 && !isCorrupt) {
>         // If the block is corrupted, it means all its available replicas are
>         // corrupted. We don't mark it as missing given these available replicas
>         // might still be accessible as the block might be incorrectly marked as
>         // corrupted by client machines.
> {code}
> While in other reports even if all the replicas are corrupted , block is been considered
as missed.
> Please provide your thoughts : can we make missing block count consistent across all
the reports same as implemented for fsck?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message