cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sylvain Lebresne (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-9738) Migrate key-cache to be fully off-heap
Date Fri, 11 Sep 2015 09:01:46 GMT


Sylvain Lebresne commented on CASSANDRA-9738:

To put your mind at ease, I'm not particularly attached to any of the changes from CASSANDRA-10232.
The goal of that ticket was to make very simple change that, given how we used RIE at the
time, provided hopefully clear gains (not only for performance btw, getting smaller files
on disk is a nice thing by itself). But if the uses of RIE change and other trade-offs makes
more sense, so be it. Of course, the more we have time to validate those choices with benchmarks,
the better.

bq. I am questioning this on the grounds that it requires walking the entire partition index
and doing work even to random access?

If you mean that the cost of iterating over the index to recompute the offsets could outweight
the gain of reading less data, then you could be right but my hunch is that this probably
doesn't make much measurable difference in practice. So the goal was more to save some space
on disk than anything else (it's not hugely important, but it's a nice to have). But anyway,
I'll remark that even if we want to avoid that walk, keeping both the width and offset of
each entry is still redundant. The other alternative is to keep the offset but ditch the width,
as the width can be recomputed from the current index offset and the next one (so without
a full walk). The reason I didn't went with that alternative originally is that I suspect
the code to read old entries (the backward compatibility code) will be more awkward. Anyway,
if we keep vint encoding, I guess keeping both the width and offset is not such a big deal
and I'm fine with that if that's simpler, but it's definitively possible to keep only one.

> Migrate key-cache to be fully off-heap
> --------------------------------------
>                 Key: CASSANDRA-9738
>                 URL:
>             Project: Cassandra
>          Issue Type: Sub-task
>            Reporter: Robert Stupp
>            Assignee: Robert Stupp
>             Fix For: 3.0.0 rc1
> Key cache still uses a concurrent map on-heap. This could go to off-heap and feels doable
now after CASSANDRA-8099.
> Evaluation should be done in advance based on a POC to prove that pure off-heap counter
cache buys a performance and/or gc-pressure improvement.
> In theory, elimination of on-heap management of the map should buy us some benefit.

This message was sent by Atlassian JIRA

View raw message