cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pavel Yaskevich (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-4937) CRAR improvements (object cache + CompressionMetadata chunk offset storage moved off-heap).
Date Fri, 16 Nov 2012 04:38:16 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-4937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13498603#comment-13498603
] 

Pavel Yaskevich commented on CASSANDRA-4937:
--------------------------------------------

bq. This feels like a good compromise to me: it's a lot better than what we have now for your
use case: if an sstable is in the page cache we'd only dontneed it once gradually, instead
of repeatedly with each trySkipCache. And it doesn't take us back to the Dark Ages of totally
stomping the page cache with cold data during compaction.

I agree that it would be better than what we have, but we still would be degrading read performance
if we do so, at least that wouldn't make impact as significant as right now. Practice shows
that kernel is pretty efficient on reclaiming once used pages, when in need, and we still
have writer that skips cache. The only 100% correct place to skip cache from SSTables is after
they are compacted and before they put to the delete queue, which would give us good kernel
cache margin for a new-coming SSTable.

I have actually experimented with preheating rows is we have compaction_preheat_key_cache
enabled - for each of keys in preheat cache I WILLNEEDed first block of row in data file,
that shows some good results and doesn't overcommit the cache because it uses memory freed
after previous (compacted) sstables. 
                
> CRAR improvements (object cache + CompressionMetadata chunk offset storage moved off-heap).
> -------------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-4937
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4937
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Pavel Yaskevich
>            Assignee: Pavel Yaskevich
>             Fix For: 1.2.1
>
>         Attachments: CASSANDRA-4937.patch
>
>
> After good amount of testing on one of the clusters it was found that in order to improve
read latency we need to minimize allocation rate that compression involves, that minimizes
GC (as well as heap usage) and substantially decreases latency on read heavy workloads. 
> I have also discovered that RAR skip cache harms performance in situation when reads
are done in parallel with compaction working with relatively big SSTable files (few GB and
more). The attached patch removes possibility to skip cache from compressed files (I can also
add changes to RAR to remove skip cache functionality as a separate patch). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message