cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Ellis (JIRA)" <j...@apache.org>
Subject [jira] Commented: (CASSANDRA-1470) use direct io for compaction
Date Fri, 10 Dec 2010 16:47:03 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-1470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12970230#action_12970230
] 

Jonathan Ellis commented on CASSANDRA-1470:
-------------------------------------------

bq. This article is talking about NOREUSE flag being a no-op but we are using DONTNEED which
does work

Peter wrote a book earlier in the ticket about DONTNEED -- it sounds like it could work but
once you handle all the corner cases it may not actually be simpler than direct i/o.  I'm
open to giving it a try, though.

bq. add a method long[] pagesInPageCache() which uses the posix mincore() function to detect
the offsets of pages for this file currently in page cache

bq. use getActiveKeys() to detect which SSTables being compacted are in the os cache and make
sure the subsequent pages in the new compacted SSTable are kept in the page cache for these
keys

Let's avoid growing the scope of this ticket and make a new one for the "pre-heat sstables
post-compaction" feature.

> use direct io for compaction
> ----------------------------
>
>                 Key: CASSANDRA-1470
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1470
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Jonathan Ellis
>            Assignee: Pavel Yaskevich
>             Fix For: 0.7.1
>
>         Attachments: 1470-v2.txt, 1470.txt, CASSANDRA-1470-for-0.6.patch, CASSANDRA-1470-v10-for-0.7.patch,
CASSANDRA-1470-v11-for-0.7.patch, CASSANDRA-1470-v12-0.7.patch, CASSANDRA-1470-v2.patch, CASSANDRA-1470-v3-0.7-with-LastErrorException-support.patch,
CASSANDRA-1470-v4-for-0.7.patch, CASSANDRA-1470-v5-for-0.7.patch, CASSANDRA-1470-v6-for-0.7.patch,
CASSANDRA-1470-v7-for-0.7.patch, CASSANDRA-1470-v8-for-0.7.patch, CASSANDRA-1470-v9-for-0.7.patch,
CASSANDRA-1470.patch, use.DirectIORandomAccessFile.for.commitlog.against.1022235.patch
>
>
> When compaction scans through a group of sstables, it forces the data in the os buffer
cache being used for hot reads, which can have a dramatic negative effect on performance.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message