hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brahma Reddy Battula (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10987) Make Decommission less expensive when lot of blocks present.
Date Mon, 10 Oct 2016 16:23:20 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15562723#comment-15562723
] 

Brahma Reddy Battula commented on HDFS-10987:
---------------------------------------------

Thanks [~kihwal] for taking look..

IIUC,[~daryn] was doing finegrained locking which is big change..?  what's your view on current
patch..? 

JFYI,Tested the patch,NN was available(only first time it will take 15 to 25 sec) while running
the decommission and all the depended services(HBase,Spark,Hive..) are able to communicate.

> Make Decommission less expensive when lot of blocks present.
> ------------------------------------------------------------
>
>                 Key: HDFS-10987
>                 URL: https://issues.apache.org/jira/browse/HDFS-10987
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Brahma Reddy Battula
>            Assignee: Brahma Reddy Battula
>            Priority: Critical
>         Attachments: HDFS-10987.patch
>
>
> When user want to decommission a node which having 50M blocks +,it could hold the namesystem
lock for long time.We've seen it is taking 36 sec+. 
> As we knew during this time, Namenode will not available... As this decommission will
continuosly run till all the blocks got replicated,hence Namenode will unavailable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message