hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guangming Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-12822) HDFS unit test failure in AArch64. TestDirectoryScanner.testThrottling: Throttle is too permissive
Date Fri, 17 Nov 2017 05:26:00 GMT

     [ https://issues.apache.org/jira/browse/HDFS-12822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Guangming Zhang updated HDFS-12822:
-----------------------------------
    Description: 
Description:  Hi,  When I ran the HDFS unit test and got a failure in TestDirectoryScanner.java
test case :
TestDirectoryScanner.testThrottling:624 Throttle is too permissive
detail:
                Running org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
                Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 227.046 sec
<<< FAILURE! - in org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
                testThrottling(org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner)
 Time elapsed: 198.014 sec  <<< FAILURE!
                java.lang.AssertionError: Throttle is too permissive
                                                at org.junit.Assert.fail(Assert.java:88)
                                                at org.junit.Assert.assertTrue(Assert.java:41)
                                                at org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner.testThrottling(TestDirectoryScanner.java:624)

And below is the failure part of source code TestDirectoryScanner.java:

{code:java}
                      ...........
      while ((retries > 0) && ((ratio < 7f) || (ratio > 10f))) {
        scanner = new DirectoryScanner(dataNode, fds, conf);
        ratio = runThrottleTest(blocks);
        retries -= 1;
      }

      // Waiting should be about 9x running.
      LOG.info("RATIO: " + ratio);
      assertTrue("Throttle is too restrictive", ratio <= 10f);
      assertTrue("Throttle is too permissive", ratio >= 7f);
    ............
    private float runThrottleTest(int blocks) throws IOException {
      scanner.setRetainDiffs(true);
      scan(blocks, 0, 0, 0, 0, 0);
      scanner.shutdown();
      assertFalse(scanner.getRunStatus());
      return (float)scanner.timeWaitingMs.get() / scanner.timeRunningMs.get();
    }
  .............

{code}

The ratio in my test is 6.0578866, which is smaller than 7f in the code. So the code thrown
out an assertTrue failure.
My questions are: 
1. Why the ratio was set between 7f and 10f, is it a empirical value?
       2. The ratio is smaller than 7f in AArch64 platform, is this value within normal range?

Could anyone help? Thanks a lot. 

  was:
Description:  Hi,  When I ran the HDFS unit test and got a failure in TestDirectoryScanner.java
test case :
TestDirectoryScanner.testThrottling:624 Throttle is too permissive
detail:
                Running org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
                Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 227.046 sec
<<< FAILURE! - in org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
                testThrottling(org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner)
 Time elapsed: 198.014 sec  <<< FAILURE!
                java.lang.AssertionError: Throttle is too permissive
                                                at org.junit.Assert.fail(Assert.java:88)
                                                at org.junit.Assert.assertTrue(Assert.java:41)
                                                at org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner.testThrottling(TestDirectoryScanner.java:624)

And below is the failure part of source code TestDirectoryScanner.java:
                      ...........
   {quote}   while ((retries > 0) && ((ratio < 7f) || (ratio > 10f))) {
        scanner = new DirectoryScanner(dataNode, fds, conf);
        ratio = runThrottleTest(blocks);
        retries -= 1;
      }

      // Waiting should be about 9x running.
      LOG.info("RATIO: " + ratio);
      assertTrue("Throttle is too restrictive", ratio <= 10f);
      assertTrue("Throttle is too permissive", ratio >= 7f);
    ............
    private float runThrottleTest(int blocks) throws IOException {
      scanner.setRetainDiffs(true);
      scan(blocks, 0, 0, 0, 0, 0);
      scanner.shutdown();
      assertFalse(scanner.getRunStatus());
      return (float)scanner.timeWaitingMs.get() / scanner.timeRunningMs.get();
    }
  .............{quote}

The ratio in my test is 6.0578866, which is smaller than 7f in the code. So the code thrown
out an assertTrue failure.
My questions are: 
1. Why the ratio was set between 7f and 10f, is it a empirical value?
       2. The ratio is smaller than 7f in AArch64 platform, is this value within normal range?

Could anyone help? Thanks a lot. 


> HDFS unit test failure in AArch64. TestDirectoryScanner.testThrottling: Throttle is too
permissive
> --------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-12822
>                 URL: https://issues.apache.org/jira/browse/HDFS-12822
>             Project: Hadoop HDFS
>          Issue Type: Test
>          Components: test
>    Affects Versions: 3.1.0
>         Environment: ARMv8 AArch64, Ubuntu16.04
>            Reporter: Guangming Zhang
>            Priority: Minor
>              Labels: dtest, easyfix, maven, test
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Description:  Hi,  When I ran the HDFS unit test and got a failure in TestDirectoryScanner.java
test case :
> TestDirectoryScanner.testThrottling:624 Throttle is too permissive
> detail:
>                 Running org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
>                 Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 227.046
sec <<< FAILURE! - in org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
>                 testThrottling(org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner)
 Time elapsed: 198.014 sec  <<< FAILURE!
>                 java.lang.AssertionError: Throttle is too permissive
>                                                 at org.junit.Assert.fail(Assert.java:88)
>                                                 at org.junit.Assert.assertTrue(Assert.java:41)
>                                                 at org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner.testThrottling(TestDirectoryScanner.java:624)
> And below is the failure part of source code TestDirectoryScanner.java:
> {code:java}
>                       ...........
>       while ((retries > 0) && ((ratio < 7f) || (ratio > 10f))) {
>         scanner = new DirectoryScanner(dataNode, fds, conf);
>         ratio = runThrottleTest(blocks);
>         retries -= 1;
>       }
>       // Waiting should be about 9x running.
>       LOG.info("RATIO: " + ratio);
>       assertTrue("Throttle is too restrictive", ratio <= 10f);
>       assertTrue("Throttle is too permissive", ratio >= 7f);
>     ............
>     private float runThrottleTest(int blocks) throws IOException {
>       scanner.setRetainDiffs(true);
>       scan(blocks, 0, 0, 0, 0, 0);
>       scanner.shutdown();
>       assertFalse(scanner.getRunStatus());
>       return (float)scanner.timeWaitingMs.get() / scanner.timeRunningMs.get();
>     }
>   .............
> {code}
> The ratio in my test is 6.0578866, which is smaller than 7f in the code. So the code
thrown out an assertTrue failure.
> My questions are: 
> 1. Why the ratio was set between 7f and 10f, is it a empirical value?
>        2. The ratio is smaller than 7f in AArch64 platform, is this value within normal
range?
> Could anyone help? Thanks a lot. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message