cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Allsopp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-2850) Converting bytes to hex string is unnecessarily slow
Date Mon, 04 Jul 2011 23:06:21 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13059633#comment-13059633
] 

David Allsopp commented on CASSANDRA-2850:
------------------------------------------

Although the bytesToHex reflection hack is a bit horrible, it makes a huge difference with
really big values - I've just been trying different input sizes (with -Xmx4g -Xms4g on a 6GB
machine) and the JVM falls over with OOM at about 300MB for all the other versions, but copes
with 675MB for v4. 

With the other versions, for byte array size N, we also need at least 2N for the StringBuilder
or char[], then another 2N for the String (because the normal String constructors and methods
always do an arraycopy of the input byte[] -> 5N. 

I wonder where else in the code this sort of thing occurs...?

> Converting bytes to hex string is unnecessarily slow
> ----------------------------------------------------
>
>                 Key: CASSANDRA-2850
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2850
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>    Affects Versions: 0.7.6, 0.8.1
>            Reporter: David Allsopp
>            Priority: Minor
>             Fix For: 0.8.2
>
>         Attachments: 2850-v2.patch, BytesToHexBenchmark.java, BytesToHexBenchmark2.java,
BytesToHexBenchmark3.java, cassandra-2850a.diff
>
>
> ByteBufferUtil.bytesToHex() is unnecessarily slow - it doesn't pre-size the StringBuilder
(so several re-sizes will be needed behind the scenes) and it makes quite a few method calls
per byte.
> (OK, this may be a premature optimisation, but I couldn't resist, and it's a small change)
> Will attach patch shortly that speeds it up by about x3, plus benchmarking test.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message