hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yongjun Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-10887) Provide admin/debug tool to dump block map
Date Thu, 22 Sep 2016 07:02:21 GMT

     [ https://issues.apache.org/jira/browse/HDFS-10887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Yongjun Zhang updated HDFS-10887:
---------------------------------
    Attachment: HDFS-10887.001.patch

Upload patch rev 001.

The solution in this patch is to introduce two switches to "dfsadmin -metaSave" command, to
enable dumping out the blocks with fewer than minRepl replicas, and dumping out the full block
map.

When these switches are turned on, the patch requires safemode, because dumping out the blocks
may interfere with normal service.

This may appear a little weird, since normal "metaSave" doesn't require safemode, and it's
only required when the new switches are turned on. In case that's an issue, I also implemented
another solution that introduces a new "hdfs debug" command, which is dedicated to for this
purpose. But metaSave also log some other useful information about blocks, so it may be better
to dump them altogether.




> Provide admin/debug tool to dump block map
> ------------------------------------------
>
>                 Key: HDFS-10887
>                 URL: https://issues.apache.org/jira/browse/HDFS-10887
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs, namenode
>            Reporter: Yongjun Zhang
>            Assignee: Yongjun Zhang
>         Attachments: HDFS-10887.001.patch
>
>
> From time to time, when NN restarts, we see
> {code}
> "The reported blocks X needs additional Y blocks to reach the threshold 0.9990 of total
blocks Z. Safe mode will be turned off automatically.
> {code}
> We'd wonder what these blocks that still need block reports are, and what DNs they could
possibly be located, what happened to these DNs.
> This jira to to propose a new admin or debug tool to dump the block map info with the
blocks that have fewer than minRepl replicas.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message