cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Branimir Lambov (JIRA)" <>
Subject [jira] [Updated] (CASSANDRA-5863) In process (uncompressed) page cache
Date Tue, 22 Mar 2016 14:43:25 GMT


Branimir Lambov updated CASSANDRA-5863:
    Status: Patch Available  (was: In Progress)

Marking patch ready for review, as I believe it is worth adding to the codebase in its current
form while I continue working on replacing and improving the actual cache implementation.

The main changes the patch does are to extract rebuffering from {{\[Compressed\]RandomAccessReader}}
into a separate interface {{Rebufferer}}, and provide different rebuffering implementations
that build upon each other to provide functionality.

The simplest one is {{SimpleReadRebufferer}} which just reads from a channel, and does not
own a buffer. When cache is not enabled, a {{BufferManagingRebufferer}} is used to provide
it with a buffer only used by the specific {{RandomAccessReader}}. If cache is enabled, it
is used in combination with a shared {{ReaderCache.CachingRebufferer}} which operates cached
buffers and gives them as required to (multiple) readers, without any data copying. If rate
limitation is required, a {{LimitingRebufferer}} wrapper (specific to the reader) is applied.

In the case of compressed data, the {{SimpleReadRebufferer}} is replaced by {{CompressedRandomAccessReader.StandardRebufferer}}
or {{CompressedRandomAccessReader.MmapRebufferer}}; the stack on top of it stays the same.

Separately, uncompressed memory-mapped access is implemented in one step by {{MmapRebufferer}}
which can directly provide the mapped buffers (+ a {{LimitingRebufferer}} if required).

Most of the rebufferer stack is thread-safe and is placed in {{SegmentedFile}} to be shared
among readers.

The patch also changes {{ChecksummedDataInput}} and the other hinting-related readers to be
based on {{RebufferingInputStream}} instead of {{RandomAccessReader}} as they don't need random
access (or caching).

> In process (uncompressed) page cache
> ------------------------------------
>                 Key: CASSANDRA-5863
>                 URL:
>             Project: Cassandra
>          Issue Type: Sub-task
>            Reporter: T Jake Luciani
>            Assignee: Branimir Lambov
>              Labels: performance
>             Fix For: 3.x
> Currently, for every read, the CRAR reads each compressed chunk into a byte[], sends
it to ICompressor, gets back another byte[] and verifies a checksum.  
> This process is where the majority of time is spent in a read request.  
> Before compression, we would have zero-copy of data and could respond directly from the
> It would be useful to have some kind of Chunk cache that could speed up this process
for hot data, possibly off heap.

This message was sent by Atlassian JIRA

View raw message