hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yiqun Lin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11514) ChooseRandom can potentially be optimized
Date Mon, 13 Mar 2017 04:15:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15906838#comment-15906838
] 

Yiqun Lin commented on HDFS-11514:
----------------------------------

Thanks for this improvement, [~vagarychen]!
Would you add a unit test to verify this optimization? We can just do a add/remove node operation
after setting up initial nodes.  Then verify if the counts is updated after change. If yes,
this optimization is making sense.

> ChooseRandom can potentially be optimized
> -----------------------------------------
>
>                 Key: HDFS-11514
>                 URL: https://issues.apache.org/jira/browse/HDFS-11514
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>            Reporter: Chen Liang
>            Assignee: Chen Liang
>         Attachments: HDFS-11514.001.patch
>
>
> Based on the offline discussion, one potential improvement to the {{chooseRandomWithStorageType}}
added in HDFS-11482 is that, currently given a node, the method iterates all its children
to sum up the number of candidate datanodes. Since datanode status change is much less frequent
than block placement request. It is more efficient to get rid of this iteration check, by
probably maintaining another disk type counter map. This JIRA tracks (but not limited) this
optimization.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message