hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gopal V (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-8971) Backport: hadoop.util.PureJavaCrc32 cache hit-ratio is low for static data (HADOOP-8926)
Date Wed, 24 Oct 2012 16:04:13 GMT
Gopal V created HADOOP-8971:
-------------------------------

             Summary: Backport: hadoop.util.PureJavaCrc32 cache hit-ratio is low for static
data (HADOOP-8926)
                 Key: HADOOP-8971
                 URL: https://issues.apache.org/jira/browse/HADOOP-8971
             Project: Hadoop Common
          Issue Type: Improvement
          Components: util
    Affects Versions: 1.2.0
            Reporter: Gopal V
            Assignee: Gopal V
            Priority: Minor


Backport (HADOOP-8926) cache-aware improvements made to PureJavaCrc32 to branch-1.

With patch, observed significant improvement in crc throughput at larger chunk sizes on branch-1

java.version = 1.6.0_37
java.runtime.name = Java(TM) SE Runtime Environment
java.runtime.version = 1.6.0_37-b06
java.vm.version = 20.12-b01
java.vm.vendor = Sun Microsystems Inc.
java.vm.name = Java HotSpot(TM) 64-Bit Server VM
java.vm.specification.version = 1.0
java.specification.version = 1.6
os.arch = amd64
os.name = Linux
os.version = 3.5.0-17-generic

Performance Table (The unit is MB/sec)
|| Num Bytes ||CRC32 || PureJavaCrc32 || PureJavaCrc32+patch ||
|1 |16.548 |163.916 |122.289 |
|2 |30.046 |213.591 |246.657 |
|4 |59.853 |301.521 |322.862 |
|8 | 107.435 |491.757 |512.487 |
| 16 | 179.278 |509.099 |709.656 |
| 32 | 265.496 |651.438 |908.673 |
| 64 | 350.994 |766.566 | 1000.775 |
|128 | 410.514 |795.719 | 1073.625 |
|256 | 434.938 |828.010 | 1188.074 |
|512 | 451.997 |883.219 | 1239.929 |
| 1024 | 443.369 |902.318 | 1250.906 |
| 2048 | 427.923 |864.701 | 1228.599 |
| 4096 | 430.814 |863.258 | 1210.633 |
| 8192 | 437.079 |883.766 | 1242.756 |
|16384 | 433.349 |851.653 | 1211.161 |
|32768 | 433.188 |906.142 | 1171.551 |
|65536 | 437.420 |890.266 | 1089.157 |
| 131072 | 431.189 |869.041 | 1180.171 |
| 262144 | 430.636 |910.133 | 1212.344 |
| 524288 | 429.365 |889.867 | 1197.842 |
|1048576 | 433.012 |887.909 | 1153.719 |
|2097152 | 432.520 |861.701 | 1179.221 |
|4194304 | 439.791 |899.085 | 1178.736 |
|8388608 | 442.422 |856.197 | 1211.953 |
| 16777216 | 419.015 |866.560 | 1152.946 |


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message