hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6846) NetworkTopology#sortByDistance should give nodes higher priority, which cache the block.
Date Mon, 18 Aug 2014 21:17:19 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14101317#comment-14101317
] 

Colin Patrick McCabe commented on HDFS-6846:
--------------------------------------------

To be fair, it's pretty rare to have more than one rack-local node holding a replica, so I
don't think the change I proposed would have too much effect... except on small clusters,
or clusters using a non-default placement policy.

We should probably make this configurable at some point, so that users with massive network
bandwidth (relative to cluster size) can always prefer the cached replica...

> NetworkTopology#sortByDistance should give nodes higher priority, which cache the block.
> ----------------------------------------------------------------------------------------
>
>                 Key: HDFS-6846
>                 URL: https://issues.apache.org/jira/browse/HDFS-6846
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>    Affects Versions: 2.6.0
>            Reporter: Yi Liu
>            Assignee: Yi Liu
>
> Currently there are 3 weights:
> * local
> * same rack
> * off rack
> But if some nodes cache the block, then it's faster if client read block from these nodes.
So we should have some more weights as following:
> * local
> * cached & same rack
> * same rack
> * cached & off rack
> * off rack



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message