Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 2 Aug 2017 16:27:00 +0000 (UTC)
From: "Wei-Chiu Chuang (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.13089064.1500648357000.75254.1501691220629@Atlassian.JIRA>
In-Reply-To: <JIRA.13089064.1500648357000@Atlassian.JIRA>
References: <JIRA.13089064.1500648357000@Atlassian.JIRA> <JIRA.13089064.1500648357236@jira-lw-us.apache.org>
Subject: [jira] [Updated] (HDFS-12182) BlockManager.metaSave does not
 distinguish between "under replicated" and "missing" blocks
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 02 Aug 2017 16:27:05 -0000


     [ https://issues.apache.org/jira/browse/HDFS-12182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wei-Chiu Chuang updated HDFS-12182:
-----------------------------------
    Target Version/s: 2.9.0, 3.0.0-beta1

> BlockManager.metaSave does not distinguish between "under replicated" and "missing" blocks
> ------------------------------------------------------------------------------------------
>
>                 Key: HDFS-12182
>                 URL: https://issues.apache.org/jira/browse/HDFS-12182
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs
>            Reporter: Wellington Chevreuil
>            Assignee: Wellington Chevreuil
>            Priority: Trivial
>              Labels: newbie
>         Attachments: HDFS-12182.001.patch, HDFS-12182.002.patch, HDFS-12182.003.patch
>
>
> Currently, *BlockManager.metaSave* method (which is called by "-metasave" dfs CLI command) reports both "under replicated" and "missing" blocks under same metric *Metasave: Blocks waiting for reconstruction:* as shown on below code snippet:
> {noformat}
>    synchronized (neededReconstruction) {
>       out.println("Metasave: Blocks waiting for reconstruction: "
>           + neededReconstruction.size());
>       for (Block block : neededReconstruction) {
>         dumpBlockMeta(block, out);
>       }
>     }
> {noformat}
> *neededReconstruction* is an instance of *LowRedundancyBlocks*, which actually wraps 5 priority queues currently. 4 of these queues store different under replicated scenarios, but the 5th one is dedicated for corrupt/missing blocks. 
> Thus, metasave report may suggest some corrupt blocks are just under replicated. This can be misleading for admins and operators trying to track block missing/corruption issues, and/or other issues related to *BlockManager* metrics.
> I would like to propose a patch with trivial changes that would report corrupt blocks separately.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org