cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ariel Weisberg (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-9738) Migrate key-cache to be fully off-heap
Date Thu, 10 Sep 2015 19:35:46 GMT


Ariel Weisberg commented on CASSANDRA-9738:

For the OHC key cache we are looking at rolling back the core changes for #10232 because they
get in the way of accessing the data off heap when we want to random access fields. Also the
RIE is sampling so the value of slightly shrinking the entries seems low to me. There is a
time/space tradeoff to make since we have to binary search the indexed RIE.

Some of the other optimizations don't work like dropping fields and recalculating them because
we don't want to process the index to read it just copy the bytes. I think this also prevented
the optimization where you rebased the offset at 64k because that also requires a pass through
the data to parse.

I asked snazy to benchmark with large partitions with and without vints, but I think it's
splitting hairs either way.

It's a bad situation in general because when we cache/uncache these things we copy the entire
thing which is yucky. It will be faster this way because at least it is a straight memory
copy without an object graph or parsing.

My pony for 3.0 would be to 64-bit map the entire index file, then get 32-bit ByteBuffer slices
out of it and not use the key cache entirely, but there is a time constraint.

I think just mapping it naively might get us 80 in the 80/20 of CASSANDRA-9754.

Benedict also mentioned CASSANDRA-8931  which sounds like good next release fodder for making
the index smaller.

> Migrate key-cache to be fully off-heap
> --------------------------------------
>                 Key: CASSANDRA-9738
>                 URL:
>             Project: Cassandra
>          Issue Type: Sub-task
>            Reporter: Robert Stupp
>            Assignee: Robert Stupp
>             Fix For: 3.0.0 rc1
> Key cache still uses a concurrent map on-heap. This could go to off-heap and feels doable
now after CASSANDRA-8099.
> Evaluation should be done in advance based on a POC to prove that pure off-heap counter
cache buys a performance and/or gc-pressure improvement.
> In theory, elimination of on-heap management of the map should buy us some benefit.

This message was sent by Atlassian JIRA

View raw message