cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Goffinet (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-1902) Migrate cached pages during compaction
Date Sun, 08 May 2011 21:37:03 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-1902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13030547#comment-13030547
] 

Chris Goffinet commented on CASSANDRA-1902:
-------------------------------------------

Just to give an update, I did manage to get the hints to work, as long as I called posix_fadvise
on the previous read pair. The next problem unfortunately is that the kernel is ignoring the
FADV_RANDOM flag I call when the file is open. So read ahead still applies and has an effec
This causes problems, because a) its going to be different on many kernels b) I need to see
if its even possible to find this value without searching through /proc). I am still going
to be -1 on this patch because it's becoming more and more complex to try to manage the state
of the page cache. We have other areas in the code as well (streaming of files) that we would
need to support as well, because of the fact we aren't checking pages/and making sure we aren't
polluting the cache there. In our production environment, we already see the effects of this.


> Migrate cached pages during compaction 
> ---------------------------------------
>
>                 Key: CASSANDRA-1902
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1902
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>    Affects Versions: 0.7.1
>            Reporter: T Jake Luciani
>            Assignee: Pavel Yaskevich
>             Fix For: 1.0
>
>         Attachments: 0001-CASSANDRA-1902-cache-migration-impl-with-config-option.txt,
1902-BufferedSegmentedFile-logandsleep.txt, 1902-formatted.txt, 1902-per-column-migration-rebase2.txt,
1902-per-column-migration.txt, CASSANDRA-1902-v10-trunk-rebased.patch, CASSANDRA-1902-v3.patch,
CASSANDRA-1902-v4.patch, CASSANDRA-1902-v5.patch, CASSANDRA-1902-v6.patch, CASSANDRA-1902-v7.patch,
CASSANDRA-1902-v8.patch, CASSANDRA-1902-v9-trunk-rebased.patch, CASSANDRA-1902-v9-trunk-with-jmx.patch,
CASSANDRA-1902-v9-trunk.patch, CASSANDRA-1902-v9.patch
>
>   Original Estimate: 32h
>          Time Spent: 56h
>  Remaining Estimate: 0h
>
> Post CASSANDRA-1470 there is an opportunity to migrate cached pages from a pre-compacted
CF during the compaction process.  This is now important since CASSANDRA-1470 caches effectively
nothing.  
> For example an active CF being compacted hurts reads since nothing is cached in the new
SSTable. 
> The purpose of this ticket then is to make sure SOME data is cached from active CFs.
This can be done my monitoring which Old SSTables are in the page cache and caching active
rows in the New SStable.
> A simpler yet similar approach is described here: http://insights.oetiker.ch/linux/fadvise/

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message