Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Mon, 8 Sep 2014 20:19:29 +0000 (UTC)
From: "Dan Hendry (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12726150.1404845327000.19873.1410207569478@Atlassian.JIRA>
In-Reply-To: <JIRA.12726150.1404845327000@Atlassian.JIRA>
References: <JIRA.12726150.1404845327000@Atlassian.JIRA>
 <JIRA.12726150.1404845327278@arcas>
Subject: [jira] [Commented] (CASSANDRA-7520) Permit sorting sstables by raw
 partition key, as opposed to token
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CASSANDRA-7520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126027#comment-14126027 ] 

Dan Hendry commented on CASSANDRA-7520:
---------------------------------------

CASSANDRA-7890 Observes that LCS becomes incredibly efficient, to the point of reducing compaction overhead to near zero, if data on disk is sorted in the order it is inserted. This effect applies naturally to time series data but might also be beneficial to other types of workloads - MapReduce style processing and bulk loading (where the user can pre-sort their input data) both jump to mind. 

I for one would make the tradeoff (improved compaction and read performance at the expense of repair and bootstrapping) in a heartbeat, without reservation, for a number of our clusters. More generally, many of our workloads and data models could be changed fairly easily to take advantage of these read and compaction wins.

> Permit sorting sstables by raw partition key, as opposed to token
> -----------------------------------------------------------------
>
>                 Key: CASSANDRA-7520
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7520
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Benedict
>
> At the moment we have some counter-intuitive behaviour, which is that with a hashed partitioner (recommended) the more compacted the data is, the more randomly distributed it is amongst the file. This means that data access locality is made pretty much as bad as possible, and we rely on the OS to do its best to fix that for us with its page cache.
> [~jasobrown] mentioned this at the NGCC, but thinking on it some more it seems that many use cases may benefit from dropping the token at the storage level and sorting based on the raw key data. For workloads where nearness of key => likelihood of being coreferenced, this could improve data locality and cache hit rate dramatically. Timeseries workloads spring to mind, but I doubt this is constrained to them. Most likely any non-random access pattern could benefit. A random access pattern would most likely suffer from this scheme, as we can index more efficiently into the hashed data. However there's no reason we could not support both schemes. 


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)