hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ming Ma (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-9390) Block management for maintenance states
Date Fri, 06 Nov 2015 01:59:28 GMT
Ming Ma created HDFS-9390:

             Summary: Block management for maintenance states
                 Key: HDFS-9390
                 URL: https://issues.apache.org/jira/browse/HDFS-9390
             Project: Hadoop HDFS
          Issue Type: Sub-task
            Reporter: Ming Ma

When a node is transitioned to/stay in/transitioned out of maintenance state, we need to make
sure blocks w.r.t. that nodes are properly handled.

* When nodes are put into maintenance, it will first go to ENTERING_MAINTENANCE, and make
sure blocks are minimally replicated before the nodes are transitioned to IN_MAINTENANCE.

* Do not replica blocks when nodes are in maintenance states. Maintenance replica will remain
in BlockMaps and thus is still considered valid from block replication point of view. In other
words, putting a node to “maintenance” mode won’t trigger BlockManager to replicate
its blocks.

* Do not invalidate replicas on node under maintenance. After any file's replication factor
is reduced, NN needs to invalidate some replicas. It should exclude nodes under maintenance
in the handling.

* Do not put IN_MAINTENANCE replicas in LocatedBlock for read operation.

* Do not allocate any new block on nodes under maintenance.

* Have Balancer exclude nodes under maintenance.

* Exclude nodes under maintenance for DN cache.

This message was sent by Atlassian JIRA

View raw message