cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ariel Weisberg (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-8684) Replace usage of Adler32 with CRC32
Date Tue, 27 Jan 2015 19:12:36 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14293999#comment-14293999
] 

Ariel Weisberg commented on CASSANDRA-8684:
-------------------------------------------

And it is yet more complicated. Linux Haswell results track with OS X. Go figure! Tested on
a c4.8xlarge.

!https://docs.google.com/spreadsheets/d/1cxf-V4b8dXdz1vLb5ySUNxK09bukDHHpq79a09xHw20/pubchart?oid=2070067916&format=image!

If you are forward thinking you might stick with CRC32 as fast enough, and soon to be much
faster. The xxhash API right now also doesn't handle direct byte buffers.

And what is the magic sauce that makes CRC32Ex outperform CRC32 even though CRC32Ex is just
a derived class of CRC32?

> Replace usage of Adler32 with CRC32
> -----------------------------------
>
>                 Key: CASSANDRA-8684
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8684
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Ariel Weisberg
>            Assignee: Ariel Weisberg
>         Attachments: CRCBenchmark.java, PureJavaCrc32.java, Sample.java
>
>
> I could not find a situation in which Adler32 outperformed PureJavaCrc32 much less the
intrinsic from Java 8. For small allocations PureJavaCrc32 was much faster probably due to
the JNI overhead of invoking the native Adler32 implementation where the array has to be allocated
and copied.
> I tested on a 65w Sandy Bridge i5 running Ubuntu 14.04 with JDK 1.7.0_71 as well as a
c3.8xlarge running Ubuntu 14.04.
> I think it makes sense to stop using Adler32 when generating new checksums.
> c3.8xlarge, results are time in milliseconds, lower is better
> ||Allocation size|Adler32|CRC32|PureJavaCrc32||
> |64|47636|46075|25782|
> |128|36755|36712|23782|
> |256|31194|32211|22731|
> |1024|27194|28792|22010|
> |1048576|25941|27807|21808|
> |536870912|25957|27840|21836|
> i5
> ||Allocation size|Adler32|CRC32|PureJavaCrc32||
> |64|50539|50466|26826|
> |128|37092|38533|24553|
> |256|30630|32938|23459|
> |1024|26064|29079|22592|
> |1048576|24357|27911|22481|
> |536870912|24838|28360|22853|
> Another fun fact. Performance of the CRC32 intrinsic appears to double from Sandy Bridge
-> Haswell. Unless I am measuring something different when going from Linux/Sandy to Haswell/OS
X.
> The intrinsic/JDK 8 implementation also operates against DirectByteBuffers better and
coding against the wrapper will get that boost when run with Java 8.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message