hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinayakumar B (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-7993) Incorrect descriptions in fsck when nodes are decommissioned
Date Tue, 21 Apr 2015 04:30:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14504300#comment-14504300
] 

Vinayakumar B edited comment on HDFS-7993 at 4/21/15 4:30 AM:
--------------------------------------------------------------

In the latest patch attached, the output will change. Individual storage informations will
not be printed ex {{DatanodeInfoWithStorage\[10.19.92.116:50010,DS-13faa414-cf01-4ae7-b307-2ae87caa273c,DISK\]}}.
Thats because iterating over storages directly, not on the locations returned from located
block.

To retain the old output, Just needs following change. provided {{getStorageID()}} and {{getStorageType()}}
are public.
{code}
-            if (showRacks)
-              sb.append(NodeBase.getPath(locs[j]));
-            else
-              sb.append(locs[j]);
+            if (showRacks) {
+              sb.append(NodeBase.getPath(dnDesc));
+            } else {
+              sb.append(new DatanodeInfoWithStorage(dnDesc, storage
+                  .getStorageID(), storage.getStorageType()));
+            }
{code}


was (Author: vinayrpet):
In the latest patch attached, the output will change. Individual storage informations will
not be printed ex {{DatanodeInfoWithStorage\[10.19.92.116:50010,DS-13faa414-cf01-4ae7-b307-2ae87caa273c,DISK\]}}.
Thats because not iterating over storages directly, not on the locations returned from located
block.

To retain the old output, Just needs following change. provided {{getStorageID()}} and {{getStorageType()}}
are public.
{code}
-            if (showRacks)
-              sb.append(NodeBase.getPath(locs[j]));
-            else
-              sb.append(locs[j]);
+            if (showRacks) {
+              sb.append(NodeBase.getPath(dnDesc));
+            } else {
+              sb.append(new DatanodeInfoWithStorage(dnDesc, storage
+                  .getStorageID(), storage.getStorageType()));
+            }
{code}

> Incorrect descriptions in fsck when nodes are decommissioned
> ------------------------------------------------------------
>
>                 Key: HDFS-7993
>                 URL: https://issues.apache.org/jira/browse/HDFS-7993
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.6.0
>            Reporter: Ming Ma
>            Assignee: J.Andreina
>         Attachments: HDFS-7993.1.patch, HDFS-7993.2.patch, HDFS-7993.3.patch, HDFS-7993.4.patch,
HDFS-7993.5.patch
>
>
> When you run fsck with "-files" or "-racks", you will get something like below if one
of the replicas is decommissioned.
> {noformat}
> blk_x len=y repl=3 [dn1, dn2, dn3, dn4]
> {noformat}
> That is because in NamenodeFsck, the repl count comes from live replicas count; while
the actual nodes come from LocatedBlock which include decommissioned nodes.
> Another issue in NamenodeFsck is BlockPlacementPolicy's verifyBlockPlacement verifies
LocatedBlock that includes decommissioned nodes. However, it seems better to exclude the decommissioned
nodes in the verification; just like how fsck excludes decommissioned nodes when it check
for under replicated blocks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message