hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-6166) Improve PureJavaCrc32
Date Mon, 27 Jul 2009 19:07:14 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tsz Wo (Nicholas), SZE updated HADOOP-6166:
-------------------------------------------

    Attachment: c6166_20090727.patch

Not yet able to improve PureJavaCrc32 in my 64-bit machine but had a lot of fun last weekends.

c6166_20090727.patch: moved the codes to common (finally).  Please try it when you have time.

- 64-bit
java.version = 1.6.0_10
java.runtime.name = Java(TM) SE Runtime Environment
java.runtime.version = 1.6.0_10-b33
java.vm.version = 11.0-b15
java.vm.vendor = Sun Microsystems Inc.
java.vm.name = Java HotSpot(TM) 64-Bit Server VM
java.vm.specification.version = 1.0
java.specification.version = 1.6
os.arch = amd64
os.name = Linux
os.version = 2.6.9-55.ELsmp
||num bytes||PureJavaCrc32 MB/sec||PureJavaCrc32New MB/sec||Crc32_3_2 MB/sec||Crc32_4_3 MB/sec||Crc32_5_5
MB/sec||Crc32_6_6 MB/sec||Crc32_8_8 MB/sec||Crc32_12_12 MB/sec||
|        8 |  157.986 |  102.628 |  135.926 |  204.207 |  218.584 |  239.862 |  253.886 |
 213.072 |
|       16 |  245.381 |  214.363 |  238.798 |  202.284 |  261.246 |  207.342 |  219.935 |
 245.648 |
|       32 |  331.296 |  290.766 |  283.689 |  218.582 |  310.499 |  329.800 |  283.757 |
 273.074 |
|       64 |  405.822 |  345.573 |  325.067 |  224.623 |  344.732 |  345.010 |  311.538 |
 346.970 |
|      128 |  451.240 |  378.875 |  343.498 |  226.853 |  391.824 |  392.462 |  323.504 |
 391.573 |
|      256 |  479.728 |  410.574 |  352.432 |  226.939 |  416.448 |  396.344 |  331.537 |
 415.233 |
|      512 |  488.917 |  425.214 |  355.640 |  227.120 |  424.109 |  409.781 |  335.965 |
 427.655 |
|     1024 |  499.820 |  433.135 |  358.441 |  225.953 |  430.212 |  414.286 |  337.652 |
 431.440 |
|     2048 |  504.199 |  438.373 |  352.913 |  223.754 |  435.921 |  417.888 |  339.190 |
 438.454 |
|     4096 |  509.100 |  441.553 |  351.305 |  222.355 |  438.667 |  420.657 |  341.057 |
 441.063 |
|     8192 |  511.632 |  439.242 |  352.058 |  222.469 |  439.568 |  422.009 |  341.427 |
 447.972 |
|    16384 |  510.829 |  444.631 |  354.097 |  222.488 |  439.707 |  419.661 |  341.200 |
 451.286 |
|    32768 |  507.353 |  437.758 |  354.601 |  222.503 |  436.775 |  416.266 |  339.704 |
 449.830 |
|    65536 |  507.335 |  434.042 |  354.837 |  222.682 |  436.742 |  417.825 |  339.761 |
 449.868 |
|   131072 |  507.473 |  431.449 |  355.014 |  222.748 |  436.477 |  417.910 |  339.835 |
 449.958 |
|   262144 |  507.548 |  429.451 |  354.932 |  222.632 |  436.698 |  417.783 |  339.852 |
 449.936 |
|   524288 |  507.322 |  428.618 |  355.142 |  222.491 |  436.584 |  417.715 |  339.826 |
 450.146 |
|  1048576 |  507.148 |  428.778 |  354.769 |  222.534 |  436.506 |  417.819 |  339.830 |
 450.032 |
|  2097152 |  506.610 |  432.981 |  354.596 |  222.261 |  436.080 |  417.573 |  339.933 |
 449.623 |
|  4194304 |  504.503 |  432.501 |  352.918 |  221.669 |  432.956 |  414.626 |  337.668 |
 445.489 |
|  8388608 |  498.208 |  428.943 |  348.488 |  219.868 |  429.455 |  411.497 |  335.448 |
 440.899 |
| 16777216 |  497.184 |  423.245 |  348.105 |  219.657 |  427.288 |  410.788 |  334.992 |
 440.603 |

- 32-bit
java.version = 1.6.0_14
java.runtime.name = Java(TM) SE Runtime Environment
java.runtime.version = 1.6.0_14-b08
java.vm.version = 14.0-b16
java.vm.vendor = Sun Microsystems Inc.
java.vm.name = Java HotSpot(TM) Client VM
java.vm.specification.version = 1.0
java.specification.version = 1.6
os.arch = x86
os.name = Windows XP
os.version = 5.1
||num bytes||PureJavaCrc32 MB/sec||PureJavaCrc32New MB/sec||Crc32_3_2 MB/sec||Crc32_4_3 MB/sec||Crc32_5_5
MB/sec||Crc32_6_6 MB/sec||Crc32_8_8 MB/sec||Crc32_12_12 MB/sec||
|        8 |  192.776 |  167.821 |  174.386 |  207.658 |  184.504 |  196.480 |  222.191 |
 155.052 |
|       16 |  227.810 |  212.454 |  224.370 |  250.812 |  236.371 |  229.370 |  267.719 |
 248.392 |
|       32 |  250.230 |  239.881 |  251.688 |  280.112 |  257.619 |  270.357 |  298.951 |
 268.026 |
|       64 |  263.695 |  257.785 |  275.683 |  296.554 |  269.474 |  284.615 |  316.642 |
 319.204 |
|      128 |  270.873 |  266.500 |  286.942 |  305.744 |  282.172 |  298.166 |  325.899 |
 325.806 |
|      256 |  282.205 |  270.751 |  294.155 |  306.224 |  288.460 |  303.445 |  330.728 |
 342.959 |
|      512 |  282.529 |  270.063 |  295.103 |  309.134 |  290.043 |  300.008 |  332.325 |
 343.466 |
|     1024 |  279.710 |  273.680 |  298.489 |  308.905 |  289.993 |  305.850 |  331.706 |
 347.761 |
|     2048 |  279.753 |  274.073 |  293.911 |  304.972 |  285.538 |  308.030 |  334.459 |
 350.296 |
|     4096 |  278.830 |  275.688 |  290.520 |  302.634 |  293.201 |  308.455 |  334.498 |
 351.761 |
|     8192 |  279.518 |  274.829 |  289.088 |  299.036 |  292.383 |  305.940 |  333.512 |
 352.584 |
|    16384 |  278.609 |  251.000 |  287.964 |  303.862 |  293.782 |  308.534 |  333.397 |
 347.889 |
|    32768 |  276.124 |  272.805 |  290.125 |  300.458 |  289.993 |  306.447 |  334.985 |
 350.833 |
|    65536 |  274.212 |  273.606 |  288.872 |  303.673 |  286.457 |  307.196 |  332.591 |
 349.273 |
|   131072 |  275.371 |  272.257 |  289.985 |  303.490 |  289.126 |  303.128 |  330.630 |
 349.575 |
|   262144 |  275.607 |  273.878 |  288.080 |  302.439 |  285.862 |  304.965 |  330.775 |
 347.044 |
|   524288 |  274.578 |  270.549 |  286.745 |  299.832 |  287.063 |  304.299 |  332.160 |
 346.131 |
|  1048576 |  270.002 |  272.333 |  285.866 |  299.702 |  284.005 |  304.845 |  329.455 |
 343.377 |
|  2097152 |  268.254 |  265.650 |  285.905 |  297.428 |  286.168 |  302.749 |  329.962 |
 344.515 |
|  4194304 |  272.093 |  268.692 |  285.552 |  299.619 |  285.262 |  299.311 |  327.311 |
 338.847 |
|  8388608 |  268.156 |  265.971 |  283.967 |  291.225 |  282.974 |  301.162 |  323.698 |
 343.482 |
| 16777216 |  271.428 |  270.893 |  285.939 |  299.171 |  284.946 |  302.520 |  328.465 |
 343.694 |



> Improve PureJavaCrc32
> ---------------------
>
>                 Key: HADOOP-6166
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6166
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: util
>            Reporter: Tsz Wo (Nicholas), SZE
>            Assignee: Tsz Wo (Nicholas), SZE
>         Attachments: c6166_20090722.patch, c6166_20090722_benchmark_32VM.txt, c6166_20090722_benchmark_64VM.txt,
c6166_20090727.patch
>
>
> Got some ideas to improve CRC32 calculation.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message