hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kihwal Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10987) Make Decommission less expensive when lot of blocks present.
Date Tue, 11 Oct 2016 15:49:20 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15565805#comment-15565805

Kihwal Lee commented on HDFS-10987:

bq. IIUC,Daryn Sharp was doing finegrained locking which is big change..?
It is a decomm manager and replication monitor change. He has a POF patch, but got preempted
by other things before filing a jira for it.  

Now the patch. It looks okay in general. Just a few minor things.
// Yielding is required in case of single Node have huge number of
// blocks to process.
To be precise, the number of blocks doesn't have to be huge. It will yield if the number is
greater than the configured per-iteration-limit.
When the sleep is interrupted, it should probably not ignore. It looks like it can simply

> Make Decommission less expensive when lot of blocks present.
> ------------------------------------------------------------
>                 Key: HDFS-10987
>                 URL: https://issues.apache.org/jira/browse/HDFS-10987
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Brahma Reddy Battula
>            Assignee: Brahma Reddy Battula
>            Priority: Critical
>         Attachments: HDFS-10987.patch
> When user want to decommission a node which having 50M blocks +,it could hold the namesystem
lock for long time.We've seen it is taking 36 sec+. 
> As we knew during this time, Namenode will not available... As this decommission will
continuosly run till all the blocks got replicated,hence Namenode will unavailable.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message