cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <>
Subject [jira] [Reopened] (CASSANDRA-8434) L0 should have a separate configurable bloom filter false positive ratio
Date Fri, 22 Jul 2016 16:46:21 GMT


Benedict reopened CASSANDRA-8434:

Except that is not what I proposed.  I proposed having L0 always have a lower FP, ie that
of STCS since this is the compaction strategy used for L0. This would translate into higher
average values during overload, but not via any automation.  It's a bit anti democratic to
declare the memory burden a showstopper without any dialogue, or apparently reading the ticket.

I have little vested interest in the outcome of this decision, but as in my last interaction,
I abhor bad decision making processes; if a good process reaches a bad decision that's comparatively
fine by me.

On a database smaller than 4Tb just 3 L0 files increases server read load by 10%.  On a database
with tables smaller than 25Gb per node it is 15%.  This overload scales linearly with L0 file
count. So you only need a very small number of files in L0 to start overloading the server
by tens of percent of its normal workload, significantly lengthening the time period the server
is overloaded for and hence how large L0 grows. As a result, the *number* of BF grows larger
and you may easily reach equivalent memory pressure increase, eliminating any perceived advantage
on that front. 

Conversely, on a cluster with any appreciable amount of data stored, a few extra sstables
with larger BFs will make perhaps fractions of a percent difference to total memory consumption.

> L0 should have a separate configurable bloom filter false positive ratio
> ------------------------------------------------------------------------
>                 Key: CASSANDRA-8434
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Benedict
>             Fix For: 2.1.x
> In follow up to CASSANDRA-5371. We now perform size-tiered file selection for compaction
if L0 gets too far behind, however as far as I can tell we stick with the CF configured false
positive ratio, likely inflating substantially the number of files we visit on average until
L0 is cleaned up. Having a a different bf fp for L0 would solve this problem without introducing
any significant burden when L0 is not overloaded.

This message was sent by Atlassian JIRA

View raw message