cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-6357) Flush memtables to separate directory
Date Wed, 23 Apr 2014 21:16:24 GMT


Benedict commented on CASSANDRA-6357:

Well strictly speaking it's not _necessary_ with the latest version of 6916, as all sstablereaders
of the same table now equals() each other, but I felt a little uncomfortable getting rid of
it. The idea is(/was) that anybody using the set would see the most recent version of the
readers that were passed in. Both of the rewriters mutate the set (never simultaneously) whenever
they perform a reopen, and replace the old readers in the DataTracker so both rewriters are
always working with the actually extant copies.

I'm pretty sure it's safe to get rid of it, though, so perhaps we should as it is clearly
not intuitive.

> Flush memtables to separate directory
> -------------------------------------
>                 Key: CASSANDRA-6357
>                 URL:
>             Project: Cassandra
>          Issue Type: New Feature
>          Components: Core
>            Reporter: Patrick McFadin
>            Assignee: Jonathan Ellis
>            Priority: Minor
>              Labels: performance
>             Fix For: 2.1 beta1
>         Attachments: 6357-revert.txt, 6357-v2.txt, 6357.txt, c6357-2.1-stress-write-adj-ops-sec.png,
c6357-2.1-stress-write-latency-99th.png, c6357-2.1-stress-write-latency-median.png, c6357-stress-write-latency-99th-1.png
> Flush writers are a critical element for keeping a node healthy. When several compactions
run on systems with low performing data directories, IO becomes a premium. Once the disk subsystem
is saturated, write IO is blocked which will cause flush writer threads to backup. Since memtables
are large blocks of memory in the JVM, too much blocking can cause excessive GC over time
degrading performance. In the worst case causing an OOM.
> Since compaction is running on the data directories. My proposal is to create a separate
directory for flushing memtables. Potentially we can use the same methodology of keeping the
commit log separate and minimize disk contention against the critical function of the flushwriter.

This message was sent by Atlassian JIRA

View raw message