Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Message-ID: <30115013.40091291916763344.JavaMail.jira@thor>
Date: Thu, 9 Dec 2010 12:46:03 -0500 (EST)
From: "Peter Schuller (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Subject: [jira] Commented: (CASSANDRA-1470) use direct io for compaction
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CASSANDRA-1470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12969827#action_12969827 ] 

Peter Schuller commented on CASSANDRA-1470:
-------------------------------------------

Note that at minimum the amount written per direct write has to be high enough that the seek overhead of constituent disks becomes irrelevant (and keeping in mind that there may be multiple spindles under a RAID controller). In addition on a loaded system there will be additional latency resulting from the I/O requests queueing, which increases the demands for the size of individual writes. I would expect a sensible "chunk" size to be several megabyte at minimum for writes.

The other trade-off for reads (and writes if not using battery-backed write caching in a RAID controller) is that too large "chunk:s" will be detrimental to latency of live traffic. I exclude writes on battery backed controllers under the assumption that the chunk size is well below cache size in the controller. However, on the other hand there is a throttling issue here so unless rate limiting is applied you would see detrimental affects with a RAID controller too since it is going to eat your writes until it becomes full, at which point submission of a huge write will probably again have the detrimental effects on latency.

> use direct io for compaction
> ----------------------------
>
>                 Key: CASSANDRA-1470
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1470
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Jonathan Ellis
>            Assignee: Pavel Yaskevich
>             Fix For: 0.7.1
>
>         Attachments: 1470-v2.txt, 1470.txt, CASSANDRA-1470-for-0.6.patch, CASSANDRA-1470-v10-for-0.7.patch, CASSANDRA-1470-v11-for-0.7.patch, CASSANDRA-1470-v12-0.7.patch, CASSANDRA-1470-v2.patch, CASSANDRA-1470-v3-0.7-with-LastErrorException-support.patch, CASSANDRA-1470-v4-for-0.7.patch, CASSANDRA-1470-v5-for-0.7.patch, CASSANDRA-1470-v6-for-0.7.patch, CASSANDRA-1470-v7-for-0.7.patch, CASSANDRA-1470-v8-for-0.7.patch, CASSANDRA-1470-v9-for-0.7.patch, CASSANDRA-1470.patch, use.DirectIORandomAccessFile.for.commitlog.against.1022235.patch
>
>
> When compaction scans through a group of sstables, it forces the data in the os buffer cache being used for hot reads, which can have a dramatic negative effect on performance.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.