cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Branimir Lambov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-8464) Support direct buffer decompression for reads
Date Wed, 31 Dec 2014 17:17:14 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-8464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14262311#comment-14262311
] 

Branimir Lambov commented on CASSANDRA-8464:
--------------------------------------------

Sorry for the late review. A couple more things to clean up:

- Does it work with {{DeflateCompressor}} now? AFAICS it will fail as the input buffer will
be a mapped one. Am I missing something? Could this be tested? If it does not work, perhaps
{{CompressedRandomAccessReader}} should choose to not use memory-mapping for some compressors?

- I'd prefer {{LZ4Compressor.uncompress()}} to not make any changes to the position or limit
of either buffer. Currently you are modifying the input's position by copying from the input
buffer into an array to read the length, which is not necessary as you can read bytes at a
given offset in the buffer directly.
 
- Could you clean up the imports of {{sun.nio.ch.DirectBuffer}} that were introduced in the
previous version?


> Support direct buffer decompression for reads
> ---------------------------------------------
>
>                 Key: CASSANDRA-8464
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8464
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: T Jake Luciani
>            Assignee: T Jake Luciani
>              Labels: performance
>             Fix For: 3.0
>
>         Attachments: compression_direct.png
>
>
> Currently when we read a compressed sstable we copy the data on heap then send it to
be de-compressed to another on heap buffer (albeit pooled).
> But now both snappy and lz4 (with CASSANDRA-7039) allow decompression of direct byte
buffers.   This lets us mmap the data and decompress completely off heap (and avoids moving
bytes over JNI).
> One issue is performing the checksum offheap but the Adler32 does support in java 8 (it's
also in java 7 but marked private?!)
> This change yields a > 10% boost in read performance on cstar.  Locally I see upto
30% improvement.
> http://cstar.datastax.com/graph?stats=5ebcdd70-816b-11e4-aed6-42010af0688f&metric=op_rate&operation=2_read&smoothing=1&show_aggregates=true&xmin=0&xmax=200.09&ymin=0&ymax=135908.3



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message