hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-11896) Non-dfsUsed will be doubled on dead node re-registration in branch-2.7.
Date Thu, 20 Jul 2017 01:13:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16093590#comment-16093590
] 

Konstantin Shvachko edited comment on HDFS-11896 at 7/20/17 1:12 AM:
---------------------------------------------------------------------

Hey [~brahmareddy],
* in your trunk patch you correctly reset nonDfsUsed to 0, but not other fields. I was thinking
we could factor out the part of {{updateHeartbeatState()}} that sets the "total" fields excluding
the update times and use it in {{resetBlocks()}}. That way we know all fields are updated
even if somebody adds new ones.
* In the 2.7 patch you follow the HDFS-9034 path introducing {{updateDnStat()}}, which will
cause test failures as HDFS-9034 did. I recommend instead to swap lines in {{HeartbeatManager.register()}}
without modifying {{addDatanode()}}.
Like this:
{code}
  synchronized void register(final DatanodeDescriptor d) {
    if (!d.isAlive) {
-      addDatanode(d);
-      d.updateHeartbeatState(StorageReport.EMPTY_ARRAY, 0L, 0L, 0, 0, null);
+      d.updateHeartbeatState(StorageReport.EMPTY_ARRAY, 0L, 0L, 0, 0, null);
+      addDatanode(d);
    }
  }
{code}


was (Author: shv):
Hey [~brahmareddy],
* in your trunk patch you correctly reset nonDfsUsed to 0, but not other fields. I was thinking
we could factor out the part of {{updateHeartbeatState()}} that sets the "total" fields excluding
the update times and use it in {{resetBlocks()}}. That way we know all fields are updated
even if somebody adds new ones.
* In the 2.7 patch you follow the HDFS-9034 path introducing {{updateDnStat()}}, which will
cause test failures as HDFS-9034 did. I recommend instead to swap lines in {{HeartbeatManager.register()}}
without modifying {{addDatanode()}}.

> Non-dfsUsed will be doubled on dead node re-registration in branch-2.7.
> -----------------------------------------------------------------------
>
>                 Key: HDFS-11896
>                 URL: https://issues.apache.org/jira/browse/HDFS-11896
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.7.3
>            Reporter: Brahma Reddy Battula
>            Assignee: Brahma Reddy Battula
>              Labels: release-blocker
>         Attachments: HDFS-11896-002.patch, HDFS-11896-branch-2.7-001.patch, HDFS-11896-branch-2.7-002.patch,
HDFS-11896.patch
>
>
>  *Scenario:* 
> i)Make you sure you've non-dfs data.
> ii) Stop Datanode
> iii) wait it becomes dead
> iv) now restart and check the non-dfs data



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message