cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcus Eriksson (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-11412) Many sstablescanners opened during repair
Date Tue, 12 Apr 2016 11:54:25 GMT


Marcus Eriksson commented on CASSANDRA-11412:

My thinking is that for non-LCS workloads "all" sstables cover the whole range - it should
be quite rare that we have any benefit from doing something like you suggest - reason being
that with other compaction strategies we never split based on range - "all" sstables cover
the whole range.

Now, this is not entirely true of course, say temporarily after a bootstrap or a repair we
could have many non-overlapping sstables, but the gain should be small and during a short
time span because:
* we never compact more than 32 sstables together so during compaction we open at most 32
* repair opens all sstables, but in the incremental world we only open unrepaired sstables,
while the many streamed in sstables after a repair are already repaired.

Does that make sense? I think we should do the improvement, but in another ticket as it is
not as urgent as this

> Many sstablescanners opened during repair
> -----------------------------------------
>                 Key: CASSANDRA-11412
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Marcus Eriksson
>            Assignee: Marcus Eriksson
>             Fix For: 3.0.x, 3.x
> Since CASSANDRA-5220 we open [one sstablescanner per range per sstable|].
If compaction gets behind and you are running vnodes with 256 tokens and RF3, this could become
a problem (ie, {{768 * number of sstables}} scanners)
> We could probably refactor this similar to the way we handle scanners with LCS - only
open the scanner once we need it

This message was sent by Atlassian JIRA

View raw message