hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Íñigo Goiri (JIRA) <j...@apache.org>
Subject [jira] [Commented] (HADOOP-16161) NetworkTopology#getWeightUsingNetworkLocation return unexpected result
Date Thu, 04 Apr 2019 04:47:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-16161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16809502#comment-16809502

Íñigo Goiri commented on HADOOP-16161:

The main thing with {{assertEquals()}} is that when debugging failed unit tests from others,
it will tell you the expected and actual in the assert error in the wrong way.
In this way is easier to get assertion errors.
[^HADOOP-16161.009.patch] LGTM.
+1 pending Yetus.

> NetworkTopology#getWeightUsingNetworkLocation return unexpected result
> ----------------------------------------------------------------------
>                 Key: HADOOP-16161
>                 URL: https://issues.apache.org/jira/browse/HADOOP-16161
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: net
>            Reporter: He Xiaoqiao
>            Assignee: He Xiaoqiao
>            Priority: Major
>         Attachments: HADOOP-16161.001.patch, HADOOP-16161.002.patch, HADOOP-16161.003.patch,
HADOOP-16161.004.patch, HADOOP-16161.005.patch, HADOOP-16161.006.patch, HADOOP-16161.007.patch,
HADOOP-16161.008.patch, HADOOP-16161.009.patch
> Consider the following scenario:
> 1. there are 4 slaves and topology like:
> Rack: /IDC/RACK1
>    hostname1
>    hostname2
> Rack: /IDC/RACK2
>    hostname3
>    hostname4
> 2. Reader from hostname1, and calculate weight between reader and [hostname1, hostname3,
hostname4] by #getWeight, and their corresponding values are [0,4,4]
> 3. Reader from client which is not in the topology, and in the same IDC but in none rack
of the topology, and calculate weight between reader and [hostname1, hostname3, hostname4]
by #getWeightUsingNetworkLocation, and their corresponding values are [2,2,2]
> 4. Other different Reader can get the similar results.
> The weight result for case #3 is obviously not the expected value, the truth is [4,4,4].
this issue may cause reader not really following arrange: local -> local rack -> remote
> After dig the detailed implement, the root cause is #getWeightUsingNetworkLocation only
calculate distance between Racks rather than hosts.
> I think we should add constant 2 to correct the weight of #getWeightUsingNetworkLocation.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message