hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lei (Eddy) Xu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9390) Block management for maintenance states
Date Fri, 14 Oct 2016 19:05:20 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15576198#comment-15576198
] 

Lei (Eddy) Xu commented on HDFS-9390:
-------------------------------------

Hi, [~mingma]

The latest patch needs to rebase to trunk. But the code LGTM. +1 pending the rebase.

Thanks for the great effort!

> Block management for maintenance states
> ---------------------------------------
>
>                 Key: HDFS-9390
>                 URL: https://issues.apache.org/jira/browse/HDFS-9390
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Ming Ma
>            Assignee: Ming Ma
>         Attachments: HDFS-9390-2.patch, HDFS-9390-3.patch, HDFS-9390-4.patch, HDFS-9390.patch
>
>
> When a node is transitioned to/stay in/transitioned out of maintenance state, we need
to make sure blocks w.r.t. that nodes are properly handled.
> * When nodes are put into maintenance, it will first go to ENTERING_MAINTENANCE, and
make sure blocks are minimally replicated before the nodes are transitioned to IN_MAINTENANCE.
> * Do not replica blocks when nodes are in maintenance states. Maintenance replica will
remain in BlockMaps and thus is still considered valid from block replication point of view.
In other words, putting a node to “maintenance” mode won’t trigger BlockManager to replicate
its blocks.
> * Do not invalidate replicas on node under maintenance. After any file's replication
factor is reduced, NN needs to invalidate some replicas. It should exclude nodes under maintenance
in the handling.
> * Do not put IN_MAINTENANCE replicas in LocatedBlock for read operation.
> * Do not allocate any new block on nodes under maintenance.
> * Have Balancer exclude nodes under maintenance.
> * Exclude nodes under maintenance for DN cache.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message