aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aurora ReviewBot <wfar...@apache.org>
Subject Re: Review Request 55284: Benchmarking SLA Algorithms
Date Tue, 10 Jan 2017 09:02:50 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55284/#review161046
-----------------------------------------------------------



Master (d4ebb56) is red with this patch.
  ./build-support/jenkins/build.sh


:commons:generateThriftResources
:commons:processResources
:commons:classes
:commons:jar
:compileJava/home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/log/WriteAheadStorage.java:74:
Note: Wrote forwarder org.apache.aurora.scheduler.storage.log.WriteAheadStorageForwarder
@Forward({
^
Note: Writing file:/home/jenkins/jenkins-slave/workspace/AuroraBot/dist/classes/main/org/apache/aurora/common/args/apt/cmdline.arg.info.txt.2
Note: Writing file:/home/jenkins/jenkins-slave/workspace/AuroraBot/dist/classes/main/META-INF/compiler/resource-mappings/org.apache.aurora.common.args.apt.CmdLineProcessor

:generateBuildProperties
:processResources
:classes
:jar
:startScripts
:distTar
:distZip
:assemble
:compileJmhJavaNote: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/jmh/java/org/apache/aurora/benchmark/fakes/FakeSchedulerDriver.java
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

:processJmhResources UP-TO-DATE
:jmhClasses
:checkstyleJmh[ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/jmh/java/org/apache/aurora/benchmark/Tasks.java:18:8:
Unused import - java.util.Optional. [UnusedImports]
[ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/jmh/java/org/apache/aurora/benchmark/Tasks.java:107:54:
'{' is not preceded with whitespace. [WhitespaceAround]
 FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':checkstyleJmh'.
> Checkstyle rule violations were found. See the report at: file:///home/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/checkstyle/jmh.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to
get more log output.

BUILD FAILED

Total time: 1 mins 8.49 secs


I will refresh this build result if you post a review containing "@ReviewBot retry"

- Aurora ReviewBot


On Jan. 10, 2017, 9 a.m., Reza Motamedi wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55284/
> -----------------------------------------------------------
> 
> (Updated Jan. 10, 2017, 9 a.m.)
> 
> 
> Review request for Aurora, Joshua Cohen, Mehrdad Nurolahzade, and Santhosh Kumar Shanmugham.
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> # Adding SLA benchmarks benchmarks 
> This RB is related to [AURORA-1854](https://issues.apache.org/jira/browse/AURORA-1854).

> The goal is to add JMH benchmarks for SlaCalculation and then other metric calculators.
> 
> 
> Diffs
> -----
> 
>   src/jmh/java/org/apache/aurora/benchmark/SlaAlgorithmBenchmark.java PRE-CREATION 
>   src/jmh/java/org/apache/aurora/benchmark/Tasks.java 3aabc5ef222a97d8ebde693c52030a66bca340d2

>   src/main/java/org/apache/aurora/scheduler/sla/SlaAlgorithm.java 5d8d5bd8f705770979f284d26d2e932aabe707e5

> 
> Diff: https://reviews.apache.org/r/55284/diff/
> 
> 
> Testing
> -------
> 
> the content of <./aurora/dist/reports/jmh/human.txt>
> 
> ```
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA
> # Parameters: (numTasks = 1000)
> 
> # Run progress: 0.00% complete, ETA 00:07:00
> # Fork: 1 of 1
> # Warmup Iteration   1: 5141.403 ops/s
> Iteration   1: 5514.591 ops/s
> Iteration   2: 5508.875 ops/s
> Iteration   3: 5594.224 ops/s
> Iteration   4: 5467.605 ops/s
> Iteration   5: 5619.229 ops/s
> 
> 
> Result "measureMTTA":
>   5540.905 ±(99.9%) 244.059 ops/s [Average]
>   (min, avg, max) = (5467.605, 5540.905, 5619.229), stdev = 63.381
>   CI (99.9%): [5296.846, 5784.963] (assumes normal distribution)
> 
> 
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA
> # Parameters: (numTasks = 10000)
> 
> # Run progress: 8.33% complete, ETA 00:06:34
> # Fork: 1 of 1
> # Warmup Iteration   1: 64.993 ops/s
> Iteration   1: 61.762 ops/s
> Iteration   2: 64.468 ops/s
> Iteration   3: 65.047 ops/s
> Iteration   4: 63.199 ops/s
> Iteration   5: 64.412 ops/s
> 
> 
> Result "measureMTTA":
>   63.778 ±(99.9%) 5.053 ops/s [Average]
>   (min, avg, max) = (61.762, 63.778, 65.047), stdev = 1.312
>   CI (99.9%): [58.725, 68.830] (assumes normal distribution)
> 
> 
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA
> # Parameters: (numTasks = 100000)
> 
> # Run progress: 16.67% complete, ETA 00:06:00
> # Fork: 1 of 1
> # Warmup Iteration   1: 0.490 ops/s
> Iteration   1: 0.556 ops/s
> Iteration   2: 0.503 ops/s
> Iteration   3: 0.502 ops/s
> Iteration   4: 0.576 ops/s
> Iteration   5: 0.591 ops/s
> 
> 
> Result "measureMTTA":
>   0.546 ±(99.9%) 0.159 ops/s [Average]
>   (min, avg, max) = (0.502, 0.546, 0.591), stdev = 0.041
>   CI (99.9%): [0.387, 0.705] (assumes normal distribution)
> 
> 
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA
> # Parameters: (numTasks = 200000)
> 
> # Run progress: 25.00% complete, ETA 00:05:40
> # Fork: 1 of 1
> # Warmup Iteration   1: 0.099 ops/s
> Iteration   1: 0.130 ops/s
> Iteration   2: 0.146 ops/s
> Iteration   3: 0.117 ops/s
> Iteration   4: 0.145 ops/s
> Iteration   5: 0.144 ops/s
> 
> 
> Result "measureMTTA":
>   0.137 ±(99.9%) 0.049 ops/s [Average]
>   (min, avg, max) = (0.117, 0.137, 0.146), stdev = 0.013
>   CI (99.9%): [0.088, 0.185] (assumes normal distribution)
> 
> 
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR
> # Parameters: (numTasks = 1000)
> 
> # Run progress: 33.33% complete, ETA 00:05:28
> # Fork: 1 of 1
> # Warmup Iteration   1: 4184.816 ops/s
> Iteration   1: 4274.550 ops/s
> Iteration   2: 2630.657 ops/s
> Iteration   3: 3168.746 ops/s
> Iteration   4: 4054.913 ops/s
> Iteration   5: 3653.909 ops/s
> 
> 
> Result "measureMTTR":
>   3556.555 ±(99.9%) 2569.449 ops/s [Average]
>   (min, avg, max) = (2630.657, 3556.555, 4274.550), stdev = 667.278
>   CI (99.9%): [987.106, 6126.004] (assumes normal distribution)
> 
> 
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR
> # Parameters: (numTasks = 10000)
> 
> # Run progress: 41.67% complete, ETA 00:04:40
> # Fork: 1 of 1
> # Warmup Iteration   1: 39.150 ops/s
> Iteration   1: 56.293 ops/s
> Iteration   2: 59.521 ops/s
> Iteration   3: 46.747 ops/s
> Iteration   4: 55.249 ops/s
> Iteration   5: 52.481 ops/s
> 
> 
> Result "measureMTTR":
>   54.058 ±(99.9%) 18.486 ops/s [Average]
>   (min, avg, max) = (46.747, 54.058, 59.521), stdev = 4.801
>   CI (99.9%): [35.572, 72.544] (assumes normal distribution)
> 
> 
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR
> # Parameters: (numTasks = 100000)
> 
> # Run progress: 50.00% complete, ETA 00:03:57
> # Fork: 1 of 1
> # Warmup Iteration   1: 0.448 ops/s
> Iteration   1: 0.482 ops/s
> Iteration   2: 0.475 ops/s
> Iteration   3: 0.353 ops/s
> Iteration   4: 0.478 ops/s
> Iteration   5: 0.513 ops/s
> 
> 
> Result "measureMTTR":
>   0.460 ±(99.9%) 0.238 ops/s [Average]
>   (min, avg, max) = (0.353, 0.460, 0.513), stdev = 0.062
>   CI (99.9%): [0.223, 0.698] (assumes normal distribution)
> 
> 
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR
> # Parameters: (numTasks = 200000)
> 
> # Run progress: 58.33% complete, ETA 00:03:23
> # Fork: 1 of 1
> # Warmup Iteration   1: 0.085 ops/s
> Iteration   1: 0.125 ops/s
> Iteration   2: 0.133 ops/s
> Iteration   3: 0.131 ops/s
> Iteration   4: 0.129 ops/s
> Iteration   5: 0.117 ops/s
> 
> 
> Result "measureMTTR":
>   0.127 ±(99.9%) 0.025 ops/s [Average]
>   (min, avg, max) = (0.117, 0.127, 0.133), stdev = 0.006
>   CI (99.9%): [0.102, 0.152] (assumes normal distribution)
> 
> 
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS
> # Parameters: (numTasks = 1000)
> 
> # Run progress: 66.67% complete, ETA 00:02:50
> # Fork: 1 of 1
> # Warmup Iteration   1: 3914.893 ops/s
> Iteration   1: 4447.483 ops/s
> Iteration   2: 1089.132 ops/s
> Iteration   3: 1799.594 ops/s
> Iteration   4: 4622.600 ops/s
> Iteration   5: 4259.013 ops/s
> 
> 
> Result "measureMTTS":
>   3243.565 ±(99.9%) 6417.109 ops/s [Average]
>   (min, avg, max) = (1089.132, 3243.565, 4622.600), stdev = 1666.503
>   CI (99.9%): [? 0, 9660.673] (assumes normal distribution)
> 
> 
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS
> # Parameters: (numTasks = 10000)
> 
> # Run progress: 75.00% complete, ETA 00:02:05
> # Fork: 1 of 1
> # Warmup Iteration   1: 55.750 ops/s
> Iteration   1: 56.535 ops/s
> Iteration   2: 57.740 ops/s
> Iteration   3: 59.033 ops/s
> Iteration   4: 55.974 ops/s
> Iteration   5: 49.698 ops/s
> 
> 
> Result "measureMTTS":
>   55.796 ±(99.9%) 13.886 ops/s [Average]
>   (min, avg, max) = (49.698, 55.796, 59.033), stdev = 3.606
>   CI (99.9%): [41.910, 69.682] (assumes normal distribution)
> 
> 
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS
> # Parameters: (numTasks = 100000)
> 
> # Run progress: 83.33% complete, ETA 00:01:22
> # Fork: 1 of 1
> # Warmup Iteration   1: 0.484 ops/s
> Iteration   1: 0.523 ops/s
> Iteration   2: 0.544 ops/s
> Iteration   3: 0.550 ops/s
> Iteration   4: 0.518 ops/s
> Iteration   5: 0.506 ops/s
> 
> 
> Result "measureMTTS":
>   0.528 ±(99.9%) 0.071 ops/s [Average]
>   (min, avg, max) = (0.506, 0.528, 0.550), stdev = 0.018
>   CI (99.9%): [0.458, 0.599] (assumes normal distribution)
> 
> 
> # JMH 1.15 (released 102 days ago)
> # VM version: JDK 1.8.0_111, VM 25.111-b14
> # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/jre/bin/java
> # VM options: -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant
> # Warmup: 1 iterations, 10 s each
> # Measurement: 5 iterations, 5 s each
> # Timeout: 10 min per iteration
> # Threads: 1 thread, will synchronize iterations
> # Benchmark mode: Throughput, ops/time
> # Benchmark: org.apache.aurora.benchmark.SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS
> # Parameters: (numTasks = 200000)
> 
> # Run progress: 91.67% complete, ETA 00:00:41
> # Fork: 1 of 1
> # Warmup Iteration   1: 0.099 ops/s
> Iteration   1: 0.131 ops/s
> Iteration   2: 0.116 ops/s
> Iteration   3: 0.114 ops/s
> Iteration   4: 0.107 ops/s
> Iteration   5: 0.105 ops/s
> 
> 
> Result "measureMTTS":
>   0.115 ±(99.9%) 0.039 ops/s [Average]
>   (min, avg, max) = (0.105, 0.115, 0.131), stdev = 0.010
>   CI (99.9%): [0.076, 0.154] (assumes normal distribution)
> 
> 
> # Run complete. Total time: 00:08:45
> 
> Benchmark                                        (numTasks)   Mode  Cnt     Score   
  Error  Units
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA        1000  thrpt    5  5540.905 ±
 244.059  ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA       10000  thrpt    5    63.778 ±
   5.053  ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA      100000  thrpt    5     0.546 ±
   0.159  ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTA      200000  thrpt    5     0.137 ±
   0.049  ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR        1000  thrpt    5  3556.555 ±
2569.449  ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR       10000  thrpt    5    54.058 ±
  18.486  ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR      100000  thrpt    5     0.460 ±
   0.238  ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTR      200000  thrpt    5     0.127 ±
   0.025  ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS        1000  thrpt    5  3243.565 ±
6417.109  ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS       10000  thrpt    5    55.796 ±
  13.886  ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS      100000  thrpt    5     0.528 ±
   0.071  ops/s
> SlaAlgorithmBenchmark.MTTXBenchmark.measureMTTS      200000  thrpt    5     0.115 ±
   0.039  ops/s
> 
> Benchmark result is saved to dist/reports/jmh/results.txt
> 
> 
> ```
> 
> 
> Thanks,
> 
> Reza Motamedi
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message