hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hairong Kuang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5603) Improve block placement performance
Date Tue, 31 Mar 2009 22:11:50 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12694303#action_12694303
] 

Hairong Kuang commented on HADOOP-5603:
---------------------------------------

I did an experiment in a dfs cluster with 3150 node. The cluster is full with no space to
place any block. The trunk takes around 6.5s to declare failure in an effort to place a block
to 2 nodes. With the patch, it takes around 2.1s to declare failure.

> Improve block placement performance
> -----------------------------------
>
>                 Key: HADOOP-5603
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5603
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.21.0
>
>         Attachments: blockPlace.patch
>
>
> ReplicationTargetChooser chooses targets by iteratively selecting random nodes first
and then filtering good targets until the required number of targets are chosen. This code
may require selecting random nodes multiple times, thus introducing multiple traversals of
the given portion of the cluster map.  This code can be improved by traversing the portion
of the cluster map only once.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message