cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-7520) Permit sorting sstables by raw partition key, as opposed to token
Date Thu, 10 Jul 2014 06:47:05 GMT


Benedict commented on CASSANDRA-7520:

bq. I'm skeptical that this is going to help enough to be worth the extra complexity.

Fair. I'm not certain either. Especially with ng-storage, which should reduce cache pollution
by scanning fewer unrelated blocks of data. I think it will be easily tested in the non-too-distant
future though, and wanted to placeholder this whilst it occurred to me as I was mulling over
the theoretical implications of offheap storage on benchmarks.

bq. If we're not storing it in token order then we have to do a ton of random i/o on merkle
tree build and streaming. 

We can simply sequential scan and filter; once we have per-vnode storage this is probably
enough to make range repair sufficiently affordable.

> Permit sorting sstables by raw partition key, as opposed to token
> -----------------------------------------------------------------
>                 Key: CASSANDRA-7520
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Benedict
> At the moment we have some counter-intuitive behaviour, which is that with a hashed partitioner
(recommended) the more compacted the data is, the more randomly distributed it is amongst
the file. This means that data access locality is made pretty much as bad as possible, and
we rely on the OS to do its best to fix that for us with its page cache.
> [~jasobrown] mentioned this at the NGCC, but thinking on it some more it seems that many
use cases may benefit from dropping the token at the storage level and sorting based on the
raw key data. For workloads where nearness of key => likelihood of being coreferenced,
this could improve data locality and cache hit rate dramatically. Timeseries workloads spring
to mind, but I doubt this is constrained to them. Most likely any non-random access pattern
could benefit. A random access pattern would most likely suffer from this scheme, as we can
index more efficiently into the hashed data. However there's no reason we could not support
both schemes. 

This message was sent by Atlassian JIRA

View raw message