cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Branimir Lambov (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-9096) Improve ByteBuffer compression interface
Date Wed, 20 May 2015 20:24:00 GMT


Branimir Lambov commented on CASSANDRA-9096:

After running into a bug I realized reuse of ByteBuffers is not safe even with supplied positions
and lengths as the buffers' limits are still checked with index-based accesses (e.g. {{buffer.limit(1);
buffer.get(1);}} throws). To avoid any confusion about what's safe or not to do with the buffers
used in compression, I ripped off the offsets and lengths for ByteBuffer methods and made
sure their behaviour is documented and follows the standard conventions.

{{DeflateCompressor}} previously did not support operation on direct buffers at all; its preferred
buffer format (which will be the only one used ATM) is still ON_HEAP which will not go through
the buffer copying. To satisfy my curiosily I still ran a quick comparison between ON_HEAP
and OFF_HEAP buffers with it (see {{}}): decompression is up to
10% slower and compression doesn't appear to be affected at all.
Compressor DeflateCompressor ON_HEAP->ON_HEAP compress 49.825 ns/b 19.140 mb/s uncompress
3.833 ns/b 248.816 mb/s.
Compressor DeflateCompressor OFF_HEAP->OFF_HEAP compress 49.803 ns/b 19.149 mb/s uncompress
4.258 ns/b 223.997 mb/s.
The nits are applied, please take another look.

[~aboudreault]: The patch does not change anything substantial enough to show up on stress
tests, but there's no harm running them if you want to verify that.

> Improve ByteBuffer compression interface
> ----------------------------------------
>                 Key: CASSANDRA-9096
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Branimir Lambov
>            Assignee: Branimir Lambov
>             Fix For: 2.2.0 rc1
> Now that we have a few uses of compression/decompression on ByteBuffers it is time to
finalize the interface before it becomes set in stone with 2.2. The current code has some
> - The interface uses the buffers' positions and limits instead of accepting offset and
length as parameters. This necessitates that the buffers be duplicated before they can be
compressed for thread-safety, something that adds burden to the caller, is prone to being
forgotten, and we could generally do without for performance.
> - The direct/non-direct buffer support needs to be more clearly defined. The current
{{useDirectOutputByteBuffers}} is not named well.
> - If we don't want to support non-direct buffers everywhere as a fallback, we should
clearly state the decision and rationale.
> - How should {{WrappedByteBuffer}} treat direct/indirect buffers?
> - More testing is necessary as e.g. errors in {{DeflateCompressor}} were only caught
in CASSANDRA-6809.

This message was sent by Atlassian JIRA

View raw message