cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pavel Yaskevich (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-5863) In process (uncompressed) page cache
Date Tue, 26 Apr 2016 07:29:13 GMT


Pavel Yaskevich commented on CASSANDRA-5863:

bq. This is not that trivial for rebuffering – the requested length would normally be bigger
than the number of available bytes during the last {{reBuffer()}} call on the file, thus the
changes required to implement this check are too substantial to be within the scope of this

Ok, fair enough.

bq. There are other things to take into account:

Let me address the things you mentioned - all of the data is written into SSTable file in
fixed size chucks and most of the rebuffers are done (or have been done) in the granularity
of 64k or what ever compression buffer size was, since SSTable has compression parameters
we might want to have cache work on the level of sstables instead of individual files, that
way we can get access to some essential metadata. So what I was saying is that cache could
hold already decompressed 64k (or other power of 2 size) aligned buffers, either raw file
data or decompressed data based on the file; backend implementation, plugged into rebufferer,
would mmap or use regular channel read to read buffer size aligned chunks based on position
given to it, in compressed mode cache would hold decompressed buffers so it doesn't have to
share mmap'ed buffers. ReaderCache can rely on LIRS or LRU as a replacement mechanism for
aligned buffers, so each buffer is going to be reclaimed when either sstable is removed (another
reason to work closely with sstables) or replacement mechanism indicates that buffer is no
longer viable or invalidated manually. Sorry I can't provide code, so if you think that rethinking
this is not worth it, I'm fine with that.

bq. The logic is clearly defined for this round. This patch is targeted at 3.x where we can't
change sstable format in any way, and support for this format will be required long in the

I wasn't aware of that. [~jbellis] We can't modify SSTable format at all while in 3.x phrase,
even backward compatible changes in the even feature releases?

> In process (uncompressed) page cache
> ------------------------------------
>                 Key: CASSANDRA-5863
>                 URL:
>             Project: Cassandra
>          Issue Type: Sub-task
>            Reporter: T Jake Luciani
>            Assignee: Branimir Lambov
>              Labels: performance
>             Fix For: 3.x
> Currently, for every read, the CRAR reads each compressed chunk into a byte[], sends
it to ICompressor, gets back another byte[] and verifies a checksum.  
> This process is where the majority of time is spent in a read request.  
> Before compression, we would have zero-copy of data and could respond directly from the
> It would be useful to have some kind of Chunk cache that could speed up this process
for hot data, possibly off heap.

This message was sent by Atlassian JIRA

View raw message