commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bruno P. Kinoshita (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LANG-1110) Implement HashSetvBitSetTest using JMH
Date Fri, 16 Dec 2016 02:45:58 GMT

    [ https://issues.apache.org/jira/browse/LANG-1110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15753237#comment-15753237
] 

Bruno P. Kinoshita commented on LANG-1110:
------------------------------------------

As I never heard of JMH before, decided to have fun checking out the pull request locally,
and changing the HashSetvBitSetTest to use JMH and run it.

Will attach the HashSetvBitSetTest.java here. It took 54:09 minutes to run, without changing
any settings in the pom.xml (Ubuntu 16.04, Macbook i5 2.7, 8 cores, 16 GB, not virtualized).
Here's the benchmark results that appear in the console. Not sure if we could add it to the
project web site in some way. There may have some maven plug-in for that, but I couldn't find
it by just googling JMH and maven plug-in.

{noformat}
...
Iteration  19: 183.914 ns/op
Iteration  20: 184.460 ns/op

# Run progress: 97.50% complete, ETA 00:00:40
# Fork: 10 of 10
# Warmup Iteration   1: 214.977 ns/op
# Warmup Iteration   2: 189.999 ns/op
# Warmup Iteration   3: 193.562 ns/op
# Warmup Iteration   4: 182.608 ns/op
# Warmup Iteration   5: 188.630 ns/op
# Warmup Iteration   6: 181.914 ns/op
# Warmup Iteration   7: 183.420 ns/op
# Warmup Iteration   8: 184.043 ns/op
# Warmup Iteration   9: 183.083 ns/op
# Warmup Iteration  10: 181.218 ns/op
# Warmup Iteration  11: 183.786 ns/op
# Warmup Iteration  12: 182.048 ns/op
# Warmup Iteration  13: 189.763 ns/op
# Warmup Iteration  14: 182.147 ns/op
# Warmup Iteration  15: 183.410 ns/op
# Warmup Iteration  16: 187.146 ns/op
# Warmup Iteration  17: 184.401 ns/op
# Warmup Iteration  18: 183.841 ns/op
# Warmup Iteration  19: 185.179 ns/op
# Warmup Iteration  20: 182.114 ns/op
Iteration   1: 183.059 ns/op
Iteration   2: 185.323 ns/op
Iteration   3: 184.455 ns/op
Iteration   4: 186.051 ns/op
Iteration   5: 187.268 ns/op
Iteration   6: 190.621 ns/op
Iteration   7: 184.044 ns/op
Iteration   8: 183.438 ns/op
Iteration   9: 183.282 ns/op
Iteration  10: 182.034 ns/op
Iteration  11: 183.265 ns/op
Iteration  12: 183.481 ns/op
Iteration  13: 183.454 ns/op
Iteration  14: 184.774 ns/op
Iteration  15: 189.690 ns/op
Iteration  16: 188.991 ns/op
Iteration  17: 183.585 ns/op
Iteration  18: 181.612 ns/op
Iteration  19: 182.598 ns/op
Iteration  20: 184.105 ns/op


Result "timeExtractRemoveAll":
  196.003 ±(99.9%) 9.039 ns/op [Average]
  (min, avg, max) = (164.015, 196.003, 432.640), stdev = 38.270
  CI (99.9%): [186.965, 205.042] (assumes normal distribution)


# Run complete. Total time: 00:26:55

Benchmark                                Mode  Cnt    Score   Error  Units
HashSetvBitSetTest.testBitSet            avgt  200   53.333 ± 2.263  ns/op
HashSetvBitSetTest.testHashSet           avgt  200  212.623 ± 4.806  ns/op
HashSetvBitSetTest.timeBitSetRemoveAll   avgt  200  133.441 ± 7.006  ns/op
HashSetvBitSetTest.timeExtractRemoveAll  avgt  200  196.003 ± 9.039  ns/op

Benchmark result is saved to target/jmh-result.org.apache.commons.lang3.HashSetvBitSetTest.json
[INFO] 
[INFO] --- maven-jar-plugin:2.6:jar (default-jar) @ commons-lang3 ---
[INFO] Building jar: /home/kinow/Development/java/workspace/commons-lang/target/commons-lang3-3.5-SNAPSHOT.jar
[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ commons-lang3 ---
[INFO] 
[INFO] --- maven-jar-plugin:2.6:test-jar (default) @ commons-lang3 ---
[INFO] Building jar: /home/kinow/Development/java/workspace/commons-lang/target/commons-lang3-3.5-SNAPSHOT-tests.jar
[INFO] 
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ commons-lang3 ---
[INFO] Installing /home/kinow/Development/java/workspace/commons-lang/target/commons-lang3-3.5-SNAPSHOT.jar
to /home/kinow/.m2/repository/org/apache/commons/commons-lang3/3.5-SNAPSHOT/commons-lang3-3.5-SNAPSHOT.jar
[INFO] Installing /home/kinow/Development/java/workspace/commons-lang/pom.xml to /home/kinow/.m2/repository/org/apache/commons/commons-lang3/3.5-SNAPSHOT/commons-lang3-3.5-SNAPSHOT.pom
[INFO] Installing /home/kinow/Development/java/workspace/commons-lang/target/commons-lang3-3.5-SNAPSHOT-tests.jar
to /home/kinow/.m2/repository/org/apache/commons/commons-lang3/3.5-SNAPSHOT/commons-lang3-3.5-SNAPSHOT-tests.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 54:09 min
[INFO] Finished at: 2016-12-16T15:14:27+13:00
[INFO] Final Memory: 44M/720M
[INFO] ------------------------------------------------------------------------
{noformat}

I added the JMH annotations that I found in the [project examples|http://hg.openjdk.java.net/code-tools/jmh/file/tip/jmh-samples/src/main/java/org/openjdk/jmh/samples/]
and that seemed to make sense to me. Someone with experience on JMH tests can probably suggest
a better set up. 

Hope that helps
Bruno

> Implement HashSetvBitSetTest using JMH
> --------------------------------------
>
>                 Key: LANG-1110
>                 URL: https://issues.apache.org/jira/browse/LANG-1110
>             Project: Commons Lang
>          Issue Type: Test
>          Components: lang.*
>            Reporter: Benedikt Ritter
>             Fix For: Patch Needed, 3.6
>
>
> HashSetvBitSetTest currently writes to Std out, which is bad practice for unit tests.
Since HashSetvBitSetTest is really a performance test and not a unit test, we should reimplement
it using OpenJMH. The results could then be published on our website.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message