hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3218) Use multiple remote DN interfaces for block transfer
Date Mon, 09 Apr 2012 01:17:17 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13249641#comment-13249641

Todd Lipcon commented on HDFS-3218:

- I think it would make sense to add a utility function like {{DFSUtil.getRandomXferAddress(DatanodeID)}},
since you have a lot of repetition fo the {{DFSUtil.getRandom().nextInt}} stuff. Or even make
it a member function of the DatanodeID?

Otherwise looks good.

> Use multiple remote DN interfaces for block transfer
> ----------------------------------------------------
>                 Key: HDFS-3218
>                 URL: https://issues.apache.org/jira/browse/HDFS-3218
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs client
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>         Attachments: hdfs-3218.txt
> HDFS-3146 and HDFS-3216 expose multiple DN interfaces to the client. In order for clients,
in aggregate, to use multiple DN interfaces clients should pick different interfaces when
transferring blocks. Given that we cache client <-> DN connections the policy of picking
a remote interface at random for each new connection seems best (vs round robin for example).
In the future we could make the client congestion aware. We could also establish multiple
connections between the client and DN and therefore use multiple interfaces for a single block
transfer. Both of those are out of scope for this jira.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message