Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Tue, 26 Apr 2011 17:13:09 +0000 (UTC)
From: "Peter Schuller (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: 
 <318800490.3071.1303837989905.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <938819659.1774.1303804323230.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (CASSANDRA-2559) Distinguish long and short
 running compactions
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CASSANDRA-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025330#comment-13025330 ] 

Peter Schuller commented on CASSANDRA-2559:
-------------------------------------------

Uh. Sorry, I guess the AES case is adequately covered to the same extent as normal operations. Maybe the important of doing this is higher for the small CF case if you have a low gc grace, but parallel compaction shouldn't "fail" to address the issue any faster than for normal writes (other than AES being in addition to regular writes).

> Distinguish long and short running compactions
> ----------------------------------------------
>
>                 Key: CASSANDRA-2559
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2559
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Sylvain Lebresne
>            Priority: Minor
>              Labels: compaction
>
> Unless you have SSD, multi-threaded compaction is mainly here to avoid accumulating lots of newly flushed sstables while a long lasting compaction is running. But too many concurrent compactions are bad for random IO. CASSANDRA-2558 will allow to limit the number of such concurrent compactions, but choosing the right number there is not easy. If you pick too low a number, you risk accumulating "young" sstables if 2 or 3 fairly long compaction runs at the same time. On the other side, compacting multiple "small" sstables is likely to be less efficient (on a spinning disk) than compacting them serially.
> It seems to me we could have the best of both world by distinguishing long and short compactions. We could have 2 pools of thread, one for long compaction (whatever the exact definition is) and one for short ones. With this, even with one thread in each pool you would avoid most of the 'new sstable accumulation' problem while making sure you never run too many concurrent compactions (note that in theory we could stratify further than "short" and "long", but I'm not sure the benefits would out-weigh the added complexity).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira