hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sammi Chen (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDDS-699) Detect Ozone Network topology
Date Fri, 15 Mar 2019 03:30:00 GMT

    [ https://issues.apache.org/jira/browse/HDDS-699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16793279#comment-16793279
] 

Sammi Chen edited comment on HDDS-699 at 3/15/19 3:29 AM:
----------------------------------------------------------

Hi [~szetszwo], thanks very much for all your time on reviewing the patches.

the output of TestNetworkTopologyImpl fall into two categories.
 1. Purposely skip the testAncestor test when the topology only has 2 layers.
{quote}org.junit.internal.AssumptionViolatedException: got: <false>, expected: is <true>

at org.junit.Assume.assumeThat(Assume.java:95)
 at org.junit.Assume.assumeTrue(Assume.java:41)
 at org.apache.hadoop.hdds.scm.net.TestNetworkTopologyImpl.testAncestor(TestNetworkTopologyImpl.java:238)
{quote}
2. All the rest error messages are reported by TestNetworkTopologyImpl#testConcurrentAccess.
In testConcurrentAccess, all the individual tests in the class are scheduled to run currently
in different threads to test the robustness of the NetworkTopologyImpl. Operations include
add, remove, re-add and query. So some query will randomly fail when the involved node is
removed in other thread. When runing these individual tests one by one, there are no such
errors.
{quote}Exception in thread "Thread-18" java.lang.IllegalArgumentException: affinityNode /1.1.1.1
doesn't have ancestor on generation 1
 at org.apache.hadoop.hdds.scm.net.NetworkTopologyImpl.chooseNodeInternal(NetworkTopologyImpl.java:498)
 at org.apache.hadoop.hdds.scm.net.NetworkTopologyImpl.getNode(NetworkTopologyImpl.java:481)
 at org.apache.hadoop.hdds.scm.net.TestNetworkTopologyImpl.pickNodes(TestNetworkTopologyImpl.java:972)
 at org.apache.hadoop.hdds.scm.net.TestNetworkTopologyImpl.testChooseRandomWithAffinityNode(TestNetworkTopologyImpl.java:596)
 at org.apache.hadoop.hdds.scm.net.TestNetworkTopologyImpl.lambda$testConcurrentAccess$8(TestNetworkTopologyImpl.java:849)
 at java.lang.Thread.run(Thread.java:748)
{quote}


was (Author: sammi):
Hi [~szetszwo], thanks very much for all your time to review the patches. 

the output of TestNetworkTopologyImpl fall into two categories.
 1. Purposely skip the testAncestor test when the topology only has 2 layers.
{quote}org.junit.internal.AssumptionViolatedException: got: <false>, expected: is <true>

at org.junit.Assume.assumeThat(Assume.java:95)
 at org.junit.Assume.assumeTrue(Assume.java:41)
 at org.apache.hadoop.hdds.scm.net.TestNetworkTopologyImpl.testAncestor(TestNetworkTopologyImpl.java:238)
{quote}
2. All the rest error messages are reported by TestNetworkTopologyImpl#testConcurrentAccess.
In testConcurrentAccess, all the individual tests in the class are scheduled to run currently
in different threads to test the robustness of the NetworkTopologyImpl. Operations include
add, remove, re-add and query. So some query will randomly fail when the involved node is
removed in other thread. When runing these individual tests one by one, there are no such
errors. 
{quote}Exception in thread "Thread-18" java.lang.IllegalArgumentException: affinityNode /1.1.1.1
doesn't have ancestor on generation 1
 at org.apache.hadoop.hdds.scm.net.NetworkTopologyImpl.chooseNodeInternal(NetworkTopologyImpl.java:498)
 at org.apache.hadoop.hdds.scm.net.NetworkTopologyImpl.getNode(NetworkTopologyImpl.java:481)
 at org.apache.hadoop.hdds.scm.net.TestNetworkTopologyImpl.pickNodes(TestNetworkTopologyImpl.java:972)
 at org.apache.hadoop.hdds.scm.net.TestNetworkTopologyImpl.testChooseRandomWithAffinityNode(TestNetworkTopologyImpl.java:596)
 at org.apache.hadoop.hdds.scm.net.TestNetworkTopologyImpl.lambda$testConcurrentAccess$8(TestNetworkTopologyImpl.java:849)
 at java.lang.Thread.run(Thread.java:748)
{quote}

> Detect Ozone Network topology
> -----------------------------
>
>                 Key: HDDS-699
>                 URL: https://issues.apache.org/jira/browse/HDDS-699
>             Project: Hadoop Distributed Data Store
>          Issue Type: Sub-task
>            Reporter: Xiaoyu Yao
>            Assignee: Sammi Chen
>            Priority: Major
>         Attachments: HDDS-699.00.patch, HDDS-699.01.patch, HDDS-699.02.patch, HDDS-699.03.patch,
HDDS-699.04.patch, HDDS-699.05.patch, HDDS-699.06.patch, HDDS-699.07.patch, HDDS-699.08.patch
>
>
> Traditionally this has been implemented in Hadoop via script or customizable java class.
One thing we want to add here is the flexible multi-level support instead of fixed levels
like DC/Rack/NG/Node.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message