hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vikas Vishwakarma (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HADOOP-14313) Replace/improve Hadoop's byte[] comparator
Date Mon, 17 Apr 2017 02:29:41 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15970603#comment-15970603
] 

Vikas Vishwakarma edited comment on HADOOP-14313 at 4/17/17 2:29 AM:
---------------------------------------------------------------------

These microbenchmarks are calculated using JMH, 20 warmup cycles, 50 measurement cycles (total
run time 20 mins each for Hadoop & guava)

|---|Hadoop|---|---|guava|---|---|%diff|---|---|
|byte array size|min|mean|max|min|mean|max|min|mean|max|
|4|19838.782|20072.437|20090.503|24026.12|24284.021|24300.338|21|21|21|
|8|22012.932|22044.713|22051.793|22199.453|22253.173|22261.712|1|1|1|
|16|19606.912|19616.322|19649.318|21995.475|22113.443|22120.836|12|13|13|
|20|16482.859|16705.744|16776.35|18625.111|18660.355|18704.285|13|12|11|
|32|19307.22|19345.122|19352.411|21309.337|21359.051|21377.868|10|10|10|
|50|15864.759|15941.543|15953.412|18468.621|18613.202|18749.651|16|17|18|
|64|20152.624|20379.32|20397.359|21065.113|21116.799|21128.523|5|4|4|
|100|13293.668|13385.7|13403.439|15594.286|15690.369|15796.081|17|17|18|
|128|17016.59|17260.48|17278.799|17668.509|19205.199|19333.922|4|11|12|
|200|11599.469|11733.228|11755.622|14540.79|14648.077|14728.363|25|25|25|
|256|13205.591|13315.903|13326.772|14448.858|14933.008|15064.242|9|12|13|
|512|9031.652|9142.564|9149.54|10236.501|10376.17|10389.971|13|13|14|
|1024|3863.105|3864.757|3871.94|6911.951|7002.067|7009.792|79|81|81|
|2048|2129.32|2151.807|2155.381|4072.481|4085.278|4089.185|91|90|90|
|4096|1069.962|1076.303|1076.993|2319.74|2326.514|2328.69|117|116|116|
|8192|863.716|866.808|867.296|931.945|1131.406|1136.288|8|31|31|
|16384|432.556|434.158|434.698|582.37|584.294|584.852|35|35|35|


was (Author: vik.karma):
These microbenchmarks are calculated using JMH, 20 warmup cycles, 50 measurement cycles (total
run time 20 mins each for Hadoop & guava)

|---|Hadoop|---|---|guava|---|---|diff|---|---|
|byte array size|min|mean|max|min|mean|max|min|mean|max|
|19838.782|20072.437|20090.503|24026.12|24284.021|24300.338|21|21|21|
|22012.932|22044.713|22051.793|22199.453|22253.173|22261.712|1|1|1|
|19606.912|19616.322|19649.318|21995.475|22113.443|22120.836|12|13|13|
|16482.859|16705.744|16776.35|18625.111|18660.355|18704.285|13|12|11|
|19307.22|19345.122|19352.411|21309.337|21359.051|21377.868|10|10|10|
|15864.759|15941.543|15953.412|18468.621|18613.202|18749.651|16|17|18|
|20152.624|20379.32|20397.359|21065.113|21116.799|21128.523|5|4|4|
|13293.668|13385.7|13403.439|15594.286|15690.369|15796.081|17|17|18|
|17016.59|17260.48|17278.799|17668.509|19205.199|19333.922|4|11|12|
|11599.469|11733.228|11755.622|14540.79|14648.077|14728.363|25|25|25|
|13205.591|13315.903|13326.772|14448.858|14933.008|15064.242|9|12|13|
|9031.652|9142.564|9149.54|10236.501|10376.17|10389.971|13|13|14|
|3863.105|3864.757|3871.94|6911.951|7002.067|7009.792|79|81|81|
|2129.32|2151.807|2155.381|4072.481|4085.278|4089.185|91|90|90|
|1069.962|1076.303|1076.993|2319.74|2326.514|2328.69|117|116|116|
|863.716|866.808|867.296|931.945|1131.406|1136.288|8|31|31|
|432.556|434.158|434.698|582.37|584.294|584.852|35|35|35|

> Replace/improve Hadoop's byte[] comparator
> ------------------------------------------
>
>                 Key: HADOOP-14313
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14313
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: common
>            Reporter: Vikas Vishwakarma
>
> Hi,
> Recently we were looking at the Lexicographic byte array comparison in HBase. We did
microbenchmark for the byte array comparator of HADOOP, HBase Vs the latest byte array comparator
from guava  ( https://github.com/google/guava/blob/master/guava/src/com/google/common/primitives/UnsignedBytes.java#L362
) and observed that the guava main branch version is much faster. 
> Specifically we see very good improvement when the byteArraySize%8 != 0 and also for
large byte arrays. I will update the benchmark results using JMH for Hadoop vs Guava. For
the jira on HBase, please refer HBASE-17877. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message