hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-11896) Non-dfsUsed will be doubled on dead node re-registration in branch-2.7.
Date Tue, 18 Jul 2017 19:56:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092037#comment-16092037
] 

Konstantin Shvachko edited comment on HDFS-11896 at 7/18/17 7:55 PM:
---------------------------------------------------------------------

Spent some time debugging this. Found two problems:
# When heartbeat expires and {{DatanodeManager}} does {{removeDatanode()}} it does not reset
{{nonDfsUsed}}, see {{DatanodeDescriptor.resetBlocks()}}, and some other fields, which it
should.
This exists on trunk and other versions.
# During registration {{HeartbeatManager.register()}} first calls {{addDatanode()}}, which
increments {{stats}}, then resets DN usage to zeros via {{updateHeartbeatState()}}.
This was fixed by HDFS-9034, but still remains in branch-2.7.

For branch-2.7 combination of these two bugs makes nonDfsUsed counted twice on re-registration.


was (Author: shv):
Spent some time debugging this. Found two problems:
# When heartbeat expires and {{DatanodeManager}} does {{removeDatanode()}} it does not reset
{{nonDfsUsed}}, see {{DatanodeDescriptor.resetBlocks()}}, and some other fields, which it
should. This exists on trunk and other versions.
# During registration {{HeartbeatManager.register()}} first calls {{addDatanode()}}, which
increments {{stats}}, then resets DN usage to zeros via {{updateHeartbeatState()}}. This was
fixed by HDFS-9034, but still remains in branch-2.7.

For branch-2.7 combination of these two bugs makes nonDfsUsed counted twice on re-registration.

> Non-dfsUsed will be doubled on dead node re-registration in branch-2.7.
> -----------------------------------------------------------------------
>
>                 Key: HDFS-11896
>                 URL: https://issues.apache.org/jira/browse/HDFS-11896
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.7.3
>            Reporter: Brahma Reddy Battula
>            Assignee: Brahma Reddy Battula
>              Labels: release-blocker
>         Attachments: HDFS-11896-002.patch, HDFS-11896-branch-2.7-001.patch, HDFS-11896-branch-2.7-002.patch,
HDFS-11896.patch
>
>
>  *Scenario:* 
> i)Make you sure you've non-dfs data.
> ii) Stop Datanode
> iii) wait it becomes dead
> iv) now restart and check the non-dfs data



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message