hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ming Ma (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-7518) Heartbeat processing doesn't have to take FSN readLock
Date Fri, 12 Dec 2014 20:09:13 GMT
Ming Ma created HDFS-7518:

             Summary: Heartbeat processing doesn't have to take FSN readLock
                 Key: HDFS-7518
                 URL: https://issues.apache.org/jira/browse/HDFS-7518
             Project: Hadoop HDFS
          Issue Type: Sub-task
            Reporter: Ming Ma

NameNode takes global read lock when it process heartbeat RPCs from DataNodes. This increases
lock contention and could impact NN overall throughput. Given Heartbeat processing needs to
access data specific to the DataNode that invokes the RPC; it could just synchronize on the
specific DataNode and datanodeMap.

It looks like each DatanodeDescriptor already keeps its own recover blocks, replication blocks
and invalidate blocks. There are several places that needed to be changed to remove FSN lock.

As mentioned in other jiras, we need to some mechanism to reason about the correctness of
the solution.


This message was sent by Atlassian JIRA

View raw message