cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcus Eriksson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-11412) Many sstablescanners opened during repair
Date Tue, 12 Apr 2016 11:54:25 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-11412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237034#comment-15237034
] 

Marcus Eriksson commented on CASSANDRA-11412:
---------------------------------------------

My thinking is that for non-LCS workloads "all" sstables cover the whole range - it should
be quite rare that we have any benefit from doing something like you suggest - reason being
that with other compaction strategies we never split based on range - "all" sstables cover
the whole range.

Now, this is not entirely true of course, say temporarily after a bootstrap or a repair we
could have many non-overlapping sstables, but the gain should be small and during a short
time span because:
* we never compact more than 32 sstables together so during compaction we open at most 32
scanners
* repair opens all sstables, but in the incremental world we only open unrepaired sstables,
while the many streamed in sstables after a repair are already repaired.

Does that make sense? I think we should do the improvement, but in another ticket as it is
not as urgent as this

> Many sstablescanners opened during repair
> -----------------------------------------
>
>                 Key: CASSANDRA-11412
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-11412
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Marcus Eriksson
>            Assignee: Marcus Eriksson
>             Fix For: 3.0.x, 3.x
>
>
> Since CASSANDRA-5220 we open [one sstablescanner per range per sstable|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/compaction/CompactionStrategyManager.java#L374].
If compaction gets behind and you are running vnodes with 256 tokens and RF3, this could become
a problem (ie, {{768 * number of sstables}} scanners)
> We could probably refactor this similar to the way we handle scanners with LCS - only
open the scanner once we need it



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message