hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo Nicholas Sze (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10674) Rewrite the PureJavaCrc32 loop for performance improvement
Date Wed, 11 Jun 2014 02:32:02 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14027352#comment-14027352
] 

Tsz Wo Nicholas Sze commented on HADOOP-10674:
----------------------------------------------

A little more improvement.  

java.version = 1.6.0_65
java.runtime.name = Java(TM) SE Runtime Environment
java.runtime.version = 1.6.0_65-b14-462-11M4609
java.vm.version = 20.65-b04-462
java.vm.vendor = Apple Inc.
java.vm.name = Java HotSpot(TM) 64-Bit Server VM
java.vm.specification.version = 1.0
java.specification.version = 1.6
os.arch = x86_64
os.name = Mac OS X
os.version = 10.9.3

Performance Table (The unit is MB/sec)
|| Num Bytes ||    CRC32 || PureJavaCrc32 | % diff || PureJavaCrc32new | % diff | % diff ||
|          1 |    17.368 |        174.187 | 902.9% |           173.268 | 897.6% |  -0.5% |
|          2 |    34.361 |        281.842 | 720.2% |           275.534 | 701.9% |  -2.2% |
|          4 |    65.416 |        329.511 | 403.7% |           324.046 | 395.4% |  -1.7% |
|          8 |   111.836 |        624.884 | 458.7% |           674.412 | 503.0% |   7.9% |
|         16 |   177.960 |        767.225 | 331.1% |           954.177 | 436.2% |  24.4% |
|         32 |   243.528 |        926.455 | 280.4% |          1170.222 | 380.5% |  26.3% |
|         64 |   309.750 |       1039.408 | 235.6% |          1453.092 | 369.1% |  39.8% |
|        128 |   359.060 |       1106.300 | 208.1% |          1555.267 | 333.1% |  40.6% |
|        256 |   384.203 |       1128.191 | 193.6% |          1619.925 | 321.6% |  43.6% |
|        512 |   401.706 |       1108.321 | 175.9% |          1683.524 | 319.1% |  51.9% |
|       1024 |   409.730 |       1191.740 | 190.9% |          1755.902 | 328.6% |  47.3% |
|       2048 |   410.262 |       1175.336 | 186.5% |          1786.138 | 335.4% |  52.0% |
|       4096 |   417.109 |       1145.619 | 174.7% |          1768.909 | 324.1% |  54.4% |
|       8192 |   409.864 |       1138.061 | 177.7% |          1810.518 | 341.7% |  59.1% |
|      16384 |   411.105 |       1072.341 | 160.8% |          1750.499 | 325.8% |  63.2% |
|      32768 |   418.411 |       1176.763 | 181.2% |          1790.886 | 328.0% |  52.2% |
|      65536 |   413.055 |       1143.868 | 176.9% |          1792.416 | 333.9% |  56.7% |
|     131072 |   418.510 |       1053.030 | 151.6% |          1790.235 | 327.8% |  70.0% |
|     262144 |   412.248 |       1185.558 | 187.6% |          1800.560 | 336.8% |  51.9% |
|     524288 |   417.332 |       1190.188 | 185.2% |          1812.133 | 334.2% |  52.3% |
|    1048576 |   414.104 |       1119.253 | 170.3% |          1755.396 | 323.9% |  56.8% |
|    2097152 |   419.225 |       1187.693 | 183.3% |          1847.922 | 340.8% |  55.6% |
|    4194304 |   418.692 |       1171.539 | 179.8% |          1787.660 | 327.0% |  52.6% |
|    8388608 |   412.950 |       1159.336 | 180.7% |          1688.320 | 308.8% |  45.6% |
|   16777216 |   416.055 |       1199.445 | 188.3% |          1727.302 | 315.2% |  44.0% |


> Rewrite the PureJavaCrc32 loop for performance improvement
> ----------------------------------------------------------
>
>                 Key: HADOOP-10674
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10674
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: performance, util
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: Tsz Wo Nicholas Sze
>         Attachments: c10674_20140609.patch, c10674_20140609b.patch
>
>
> Below are some performance improvement opportunities performance improvement in PureJavaCrc32.
> - eliminate "off += 8; len -= 8;"
> - replace T8_x_start with hard coded constants
> - eliminate c0 - c7 local variables
> In my machine, there are 30% to 50% improvement for most of the cases.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message