Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Fri, 29 May 2015 21:42:17 +0000 (UTC)
From: "Albert P Tobey (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12833884.1432933954000.70674.1432935737657@Atlassian.JIRA>
In-Reply-To: <JIRA.12833884.1432933954000@Atlassian.JIRA>
References: <JIRA.12833884.1432933954000@Atlassian.JIRA>
 <JIRA.12833884.1432933954422@arcas>
Subject: [jira] [Commented] (CASSANDRA-9517) Switch to DTCS for hint storage
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CASSANDRA-9517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14565469#comment-14565469 ] 

Albert P Tobey commented on CASSANDRA-9517:
-------------------------------------------

My original theory was that we could use DTCS for system.hints since it has a timeseries-like table definition and let it delete whole tables when the TTLs expire. That was before I understood exactly how tombstones are used in hints. The patch seemed to help a little in testing, but I did not figure out why it seemed that way.

The forced major compaction is most of the problem when hints build up, so that's the thing that needs to be removed if at all possible. Under 100% write workload on very fast machines I was seeing system.hints compactions in excess of 100GB, which has all kinds of negative side-effects.

If there's a way we can convince any of the compaction strategies to split the wide rows across sstables (split by time window) while only merging tombstones along with subsequent cleanup, that could make hints tolerable until 3.0 takes over the world.


> Switch to DTCS for hint storage
> -------------------------------
>
>                 Key: CASSANDRA-9517
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9517
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Jeremy Hanna
>             Fix For: 2.1.6
>
>
> The DateTieredCompactionStrategy is a good choice for HintedHandoff so that we reduce the compaction load we incur when users build up hints.  [~atobey@datastax.com] and others have tried the following patch in various setups and have seen significantly less load from hint compaction.
> https://gist.github.com/tobert/c069af27e3f8840d137d
> Setting the time window to 10 minutes has shown additional improvement.
> [~krummas] do you have any feedback about this idea and/or settings?


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)