hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-12049) Recommissioning live nodes stalls the NN
Date Tue, 27 Jun 2017 15:49:00 GMT
Daryn Sharp created HDFS-12049:

             Summary: Recommissioning live nodes stalls the NN
                 Key: HDFS-12049
                 URL: https://issues.apache.org/jira/browse/HDFS-12049
             Project: Hadoop HDFS
          Issue Type: Bug
    Affects Versions: 2.6.0
            Reporter: Daryn Sharp
            Priority: Critical

A node refresh will recommission included nodes that are alive and in decommissioning or decommissioned
state.  The recommission will scan all blocks on the node, find over replicated blocks, chose
an excess, queue an invalidate.

The process is expensive and worsened by overhead of storage types (even when not in use).
 It can be especially devastating because the write lock is held for the entire node refresh.
 _Recommissioning 67 nodes with ~500k blocks/node stalled rpc services for over 4 mins._

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

View raw message