cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pavel Yaskevich (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-5661) Discard pooled readers for cold data
Date Sun, 07 Jul 2013 09:03:49 GMT


Pavel Yaskevich commented on CASSANDRA-5661:

I understand that you didn't account for any specific nuances, I just wanted to point those
things out for subscribers of that ticket...

bq. In your patch, it appears assumed that every queue as a single entry with the same size
buffer and privately Jonathan's description of the problem stated 128KB per CRAR. If the weight
is constant than they are merely a convenience mapping as it really is the number of entries.

I'm not sure if Jonathan wants to use it per CF or globally but different column families
are going to have different buffer settings especially with CRAR, my patch accounts each queue
buffer as being more or less a constant size but doesn't assume very key introduces the same

bq. The number of queues is probably going to be quite small, on the order of dozens, so the
reference cost in this case is quite small.

It's actually going to be pretty high especially with LCS (default size of 5 MB), it creates
204 files for 1GB, so it's order of thousands without taking into account compaction process.

bq. Uncontended CAS is cheap

Well if 8-32 threads (default number of threads) are going to request the same row from the
same number of files, there could be contention especially it we throw LRU in the mix. But
I'm more concerned about 1 ms polling timeout in there tho... This is why I'm concerned with
tries to make cache first class citizen instead of lucky shot especially when we don't really
need that.

> Discard pooled readers for cold data
> ------------------------------------
>                 Key: CASSANDRA-5661
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.2.1
>            Reporter: Jonathan Ellis
>            Assignee: Pavel Yaskevich
>             Fix For: 1.2.7
>         Attachments: CASSANDRA-5661.patch, DominatorTree.png, Histogram.png
> Reader pooling was introduced in CASSANDRA-4942 but pooled RandomAccessReaders are never
cleaned up until the SSTableReader is closed.  So memory use is "the worst case simultaneous
RAR we had open for this file, forever."
> We should introduce a global limit on how much memory to use for RAR, and evict old ones.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message