cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Ellis (JIRA)" <>
Subject [jira] [Assigned] (CASSANDRA-2427) Heuristic or hard cap to prevent fragmented commit logs from bringing down the server
Date Tue, 12 Apr 2011 17:30:06 GMT


Jonathan Ellis reassigned CASSANDRA-2427:

    Assignee: paul cannon

How about this:
- create a commitlog-space-cap setting; if space gets above that, flush every dirty CF in
the oldest segment and remove it
- remove the per-CF memtable_flush_after_minutes setting in favor of the above (this fixes
the flush-storm problem of all your CFs flushing at once when you configure this too low)
- while we're at it, remove the commitlog segment size config setting, nobody has ever changed
it to my knowledge in 2+ years

(I don't think we want to add an additional check for "is there enough space for a new segment"
at segment creation time; if there isn't enough space for that there probably isn't enough
to flush either and we're screwed either way. So it would be complexity w/o a purpose.)

> Heuristic or hard cap to prevent fragmented commit logs from bringing down the server
> -------------------------------------------------------------------------------------
>                 Key: CASSANDRA-2427
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Benjamin Coverston
>            Assignee: paul cannon
>              Labels: commitlog, hardening
>             Fix For: 1.0
> Widely divergent write rates on column families can cause the commit log segments to
fragment. In some cases we have seen the commit log partition overrun.
> One solution here would be to create a heuristic for segment fragmentation to trigger
a flush (commit log segments/memtable) or simply track the free disk space and force a global
flush when the disk gets to 80% capacity.

This message is automatically generated by JIRA.
For more information on JIRA, see:

View raw message