cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dan Hendry (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-7520) Permit sorting sstables by raw partition key, as opposed to token
Date Mon, 08 Sep 2014 20:19:29 GMT


Dan Hendry commented on CASSANDRA-7520:

CASSANDRA-7890 Observes that LCS becomes incredibly efficient, to the point of reducing compaction
overhead to near zero, if data on disk is sorted in the order it is inserted. This effect
applies naturally to time series data but might also be beneficial to other types of workloads
- MapReduce style processing and bulk loading (where the user can pre-sort their input data)
both jump to mind. 

I for one would make the tradeoff (improved compaction and read performance at the expense
of repair and bootstrapping) in a heartbeat, without reservation, for a number of our clusters.
More generally, many of our workloads and data models could be changed fairly easily to take
advantage of these read and compaction wins.

> Permit sorting sstables by raw partition key, as opposed to token
> -----------------------------------------------------------------
>                 Key: CASSANDRA-7520
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Benedict
> At the moment we have some counter-intuitive behaviour, which is that with a hashed partitioner
(recommended) the more compacted the data is, the more randomly distributed it is amongst
the file. This means that data access locality is made pretty much as bad as possible, and
we rely on the OS to do its best to fix that for us with its page cache.
> [~jasobrown] mentioned this at the NGCC, but thinking on it some more it seems that many
use cases may benefit from dropping the token at the storage level and sorting based on the
raw key data. For workloads where nearness of key => likelihood of being coreferenced,
this could improve data locality and cache hit rate dramatically. Timeseries workloads spring
to mind, but I doubt this is constrained to them. Most likely any non-random access pattern
could benefit. A random access pattern would most likely suffer from this scheme, as we can
index more efficiently into the hashed data. However there's no reason we could not support
both schemes. 

This message was sent by Atlassian JIRA

View raw message