impala-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthew Mulder (JIRA)" <j...@apache.org>
Subject [jira] [Created] (IMPALA-5164) BenchmarkTest.Basic test is flaky
Date Tue, 04 Apr 2017 21:57:41 GMT
Matthew Mulder created IMPALA-5164:
--------------------------------------

             Summary: BenchmarkTest.Basic test is flaky
                 Key: IMPALA-5164
                 URL: https://issues.apache.org/jira/browse/IMPALA-5164
             Project: IMPALA
          Issue Type: Bug
          Components: Infrastructure
    Affects Versions: Impala 2.8.0
            Reporter: Matthew Mulder
            Priority: Minor


The BenchmarkTest.Basic test simply compares how many times a function that memcopies 16 bytes
of data can be executed within a give time versus a function that memcopies 128 bytes of data
in the same amount of given time. Recently it had a string of failures, meaning that the function
copying more data took less time. The StopWatch class is used to do the timing. I'm assuming
this test is basically just a unit test for the StopWatch class. According to a comment in
the StopWatch class, it is inaccurate if context switching occurs. So I think the test needs
to call getrusage() before, in-between and after taking the two measurements and take the
number of context switches during each measurement into account to determine if it is fair
to compare the two measurements before deciding whether to fail. The test code is in be/src/util/benchmark-test.cc.
The test failure looks like this.{code}11:07:28       Start 55: benchmark-test
11:07:29 55/88 Test #55: benchmark-test ...................***Failed    0.54 sec
11:07:29 Turning perftools heap leak checking off
11:07:29 [==========] Running 1 test from 1 test case.
11:07:29 [----------] Global test environment set-up.
11:07:29 [----------] 1 test from BenchmarkTest
11:07:29 [ RUN      ] BenchmarkTest.Basic
11:07:29 Rate 16 Byte: 48295
11:07:29 Rate 128 Byte: 72249.1
11:07:29 /home/jenkins/Impala/be/src/util/benchmark-test.cc:67: Failure
11:07:29 Expected: (rate_copy_128) < (rate_copy_16), actual: 72249.1 vs 48295
11:07:29 [  FAILED  ] BenchmarkTest.Basic (128 ms)
11:07:29 [----------] 1 test from BenchmarkTest (128 ms total)
11:07:29 
11:07:29 [----------] Global test environment tear-down
11:07:29 [==========] 1 test from 1 test case ran. (129 ms total)
11:07:29 [  PASSED  ] 0 tests.
11:07:29 [  FAILED  ] 1 test, listed below:
11:07:29 [  FAILED  ] BenchmarkTest.Basic
11:07:29 
11:07:29  1 FAILED TEST{code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message