cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pavel Yaskevich (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-5661) Discard pooled readers for cold data
Date Sun, 07 Jul 2013 07:33:49 GMT


Pavel Yaskevich commented on CASSANDRA-5661:

[~ben.manes] I briefly looked through the code and I think there is an important component
missing - we need to weight size of the cache based on internal structure of each object allocated
into it (borrowed) as the biggest part would be in memory buffer that each instance holds,
so simple number of entries wouldn't do...

I'm worried that getResourceHandler has to go through few queues and allocation, I wanted
to avoid polling + atomic CAS on read path, which is most critical, as much as possible because
it adds additional undesired latency especially to high cardinality requests.

Also I don't think using just weak references to remove unused queues is a good idea (if I
interpreted comment at the top correctly), we need something more agressive, because ParNew+CMS
only processes those in FullGC phrase even when objects effectively die in young gen (sun/open
jdk) and G1 doesn't even have any guarantees on when they are going to be processed, not mentioning
that it requires double pass.
> Discard pooled readers for cold data
> ------------------------------------
>                 Key: CASSANDRA-5661
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.2.1
>            Reporter: Jonathan Ellis
>            Assignee: Pavel Yaskevich
>             Fix For: 1.2.7
>         Attachments: CASSANDRA-5661.patch, DominatorTree.png, Histogram.png
> Reader pooling was introduced in CASSANDRA-4942 but pooled RandomAccessReaders are never
cleaned up until the SSTableReader is closed.  So memory use is "the worst case simultaneous
RAR we had open for this file, forever."
> We should introduce a global limit on how much memory to use for RAR, and evict old ones.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message