hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lukas Majercak (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-12288) Change DN xceiver count calculation
Date Thu, 10 Aug 2017 19:49:02 GMT
Lukas Majercak created HDFS-12288:

             Summary: Change DN xceiver count calculation
                 Key: HDFS-12288
                 URL: https://issues.apache.org/jira/browse/HDFS-12288
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: datanode, hdfs
            Reporter: Lukas Majercak
            Assignee: Lukas Majercak

The problem with the ThreadGroup.activeCount() method is that the method is only a very rough
estimate, and in reality returns the total number of threads in the thread group as opposed
to the threads actually running.

In some DNs, we saw this to return 50~ for a long time, even though the actual number of DataXceiver
threads was next to none.
This is a big issue as we use the xceiverCount to make decisions on the NN for choosing replication
source DN or returning DNs to clients for R/W.

The plan is to reuse the DataNodeMetrics.dataNodeActiveXceiversCount value which only accounts
for actual number of DataXcevier threads currently running and thus represents the load on
the DN much better.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

View raw message