hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hairong Kuang (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-5603) Improve block placement performance
Date Tue, 31 Mar 2009 21:26:50 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-5603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Hairong Kuang updated HADOOP-5603:
----------------------------------

    Attachment: blockPlace.patch

Here is the patch that made the suggested change.

with the patch, both ReplicationTargetChooser#chooseRandom(int, String, List<Node>,
long, int, List<DatanodeDescriptor>) and ReplicationTarget#chooseRandom(String, List<Node>,
long, int, List<DatanodeDescriptor>) traverse every node in the given portion of the
cluster map at most once in the worst case.

> Improve block placement performance
> -----------------------------------
>
>                 Key: HADOOP-5603
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5603
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.21.0
>
>         Attachments: blockPlace.patch
>
>
> ReplicationTargetChooser chooses targets by iteratively selecting random nodes first
and then filtering good targets until the required number of targets are chosen. This code
may require selecting random nodes multiple times, thus introducing multiple traversals of
the given portion of the cluster map.  This code can be improved by traversing the portion
of the cluster map only once.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message