cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Branimir Lambov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-6726) Recycle CompressedRandomAccessReader/RandomAccessReader buffers independently of their owners, and move them off-heap when possible
Date Tue, 19 Aug 2014 07:10:18 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-6726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14101953#comment-14101953
] 

Branimir Lambov commented on CASSANDRA-6726:
--------------------------------------------

Thank you for the review.

Buffer sizes do vary, and so do the required buffer types. AFAICS compressed buffers are also
of a size we can't really pre-determine or bound and I'd much prefer not to penalize uncompressible
data further. I tried to define a flexible way to solve this which can be reused in other
parts of Cassandra that require buffers. Should I replace this with the [C]RAR-specific machinery
you propose or try to make the pool memory usage tighter?

My usual workflow is to first create a patch that addresses the design and provides baseline
working implementations, and then tweak and refine them in one or more steps to achieve the
best result, doing a lot of measurements and comparisons in the process. To do this I am going
to ignore your ThreadLocal suggestions at this stage of the patch, and apply them as refinements
later. Does this sound OK?

For ByteArrayCompressor.uncompress, I'd rather not introduce the additional complexity if
we don't plan to use it. I was actually wondering if it isn't preferable to forbid the scenarios
that require excessive copying, or at least log messages from time to time if we hit them.
What do you think?

> Recycle CompressedRandomAccessReader/RandomAccessReader buffers independently of their
owners, and move them off-heap when possible
> -----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-6726
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6726
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Benedict
>            Assignee: Branimir Lambov
>            Priority: Minor
>              Labels: performance
>             Fix For: 3.0
>
>         Attachments: cassandra-6726.patch
>
>
> Whilst CRAR and RAR are pooled, we could and probably should pool the buffers independently,
so that they are not tied to a specific sstable. It may be possible to move the RAR buffer
off-heap, and the CRAR sometimes (e.g. Snappy may possibly support off-heap buffers)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message