hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phil Yang (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade
Date Thu, 03 Dec 2015 06:18:11 GMT
Phil Yang created HDFS-9500:

             Summary: datanodesSoftwareVersions map may counting wrong when rolling upgrade
                 Key: HDFS-9500
                 URL: https://issues.apache.org/jira/browse/HDFS-9500
             Project: Hadoop HDFS
          Issue Type: Bug
    Affects Versions: 2.6.2, 2.7.1
            Reporter: Phil Yang
            Assignee: Phil Yang

While rolling upgrading, namenode's website overview will report there are two versions datanodes
in the cluster, for example, 2.6.0 has x nodes and 2.6.2 has y nodes. However, sometimes when
I stop a datanode in old version and start a new version one, namenode only increases the
number of new version but not decreases the number of old version. So the total number x+y
will be larger than the number of datanodes. Even all datanodes are upgraded, there will still
have the messages that there are several datanode in old version. And I must run hdfs dfsadmin
-refreshNodes to clear this message.

I think this issue is caused by DatanodeManager.registerDatanode. If nodeS in old version
is not alive because of shutting down, it will not pass shouldCountVersion, so the number
of old version won't be decreased. But this method only judges the status of heartbeat and
isAlive on the moment, if namenode has not been noticed and removed this node and this node
restarts in the new version, the decrementVersionCount belongs to this node will never be

So the simplest fix is that we always recounting the version map in registerDatanode since
it is not a heavy operation.

This message was sent by Atlassian JIRA

View raw message