cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Branimir Lambov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-9096) Improve ByteBuffer compression interface
Date Wed, 20 May 2015 20:24:00 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-9096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14553039#comment-14553039
] 

Branimir Lambov commented on CASSANDRA-9096:
--------------------------------------------

After running into a bug I realized reuse of ByteBuffers is not safe even with supplied positions
and lengths as the buffers' limits are still checked with index-based accesses (e.g. {{buffer.limit(1);
buffer.get(1);}} throws). To avoid any confusion about what's safe or not to do with the buffers
used in compression, I ripped off the offsets and lengths for ByteBuffer methods and made
sure their behaviour is documented and follows the standard conventions.

{{DeflateCompressor}} previously did not support operation on direct buffers at all; its preferred
buffer format (which will be the only one used ATM) is still ON_HEAP which will not go through
the buffer copying. To satisfy my curiosily I still ran a quick comparison between ON_HEAP
and OFF_HEAP buffers with it (see {{CompressorPerformance.java}}): decompression is up to
10% slower and compression doesn't appear to be affected at all.
{noformat}
Compressor DeflateCompressor ON_HEAP->ON_HEAP compress 49.825 ns/b 19.140 mb/s uncompress
3.833 ns/b 248.816 mb/s.
Compressor DeflateCompressor OFF_HEAP->OFF_HEAP compress 49.803 ns/b 19.149 mb/s uncompress
4.258 ns/b 223.997 mb/s.
{noformat}
The nits are applied, please take another look.

[~aboudreault]: The patch does not change anything substantial enough to show up on stress
tests, but there's no harm running them if you want to verify that.

> Improve ByteBuffer compression interface
> ----------------------------------------
>
>                 Key: CASSANDRA-9096
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9096
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Branimir Lambov
>            Assignee: Branimir Lambov
>             Fix For: 2.2.0 rc1
>
>
> Now that we have a few uses of compression/decompression on ByteBuffers it is time to
finalize the interface before it becomes set in stone with 2.2. The current code has some
shortcomings:
> - The interface uses the buffers' positions and limits instead of accepting offset and
length as parameters. This necessitates that the buffers be duplicated before they can be
compressed for thread-safety, something that adds burden to the caller, is prone to being
forgotten, and we could generally do without for performance.
> - The direct/non-direct buffer support needs to be more clearly defined. The current
{{useDirectOutputByteBuffers}} is not named well.
> - If we don't want to support non-direct buffers everywhere as a fallback, we should
clearly state the decision and rationale.
> - How should {{WrappedByteBuffer}} treat direct/indirect buffers?
> - More testing is necessary as e.g. errors in {{DeflateCompressor}} were only caught
in CASSANDRA-6809.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message