cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dan Kinder (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-13943) Infinite compaction of L0 SSTables in JBOD
Date Mon, 09 Oct 2017 20:55:02 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-13943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16197682#comment-16197682
] 

Dan Kinder commented on CASSANDRA-13943:
----------------------------------------

FYI:
{noformat}
data_file_directories:
    - /srv/disk1/cassandra-data
    - /srv/disk2/cassandra-data
    - /srv/disk3/cassandra-data
    - /srv/disk4/cassandra-data
    - /srv/disk5/cassandra-data
    - /srv/disk6/cassandra-data
    - /srv/disk7/cassandra-data
    - /srv/disk8/cassandra-data
    - /srv/disk9/cassandra-data
    - /srv/disk10/cassandra-data
    - /srv/disk11/cassandra-data
    - /srv/disk12/cassandra-data
{noformat}

> Infinite compaction of L0 SSTables in JBOD
> ------------------------------------------
>
>                 Key: CASSANDRA-13943
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-13943
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Compaction
>         Environment: Cassandra 3.11.0 / Centos 6
>            Reporter: Dan Kinder
>            Assignee: Marcus Eriksson
>         Attachments: debug.log
>
>
> I recently upgraded from 2.2.6 to 3.11.0.
> I am seeing Cassandra loop infinitely compacting the same data over and over. Attaching
logs.
> It is compacting two tables, one on /srv/disk10, the other on /srv/disk1. It does create
new SSTables but immediately recompacts again. Note that I am not inserting anything at the
moment, there is no flushing happening on this table (Memtable switch count has not changed).
> My theory is that it somehow thinks those should be compaction candidates. But they shouldn't
be, they are on different disks and I ran nodetool relocatesstables as well as nodetool compact.
So, it tries to compact them together, but the compaction results in the exact same 2 SSTables
on the 2 disks, because the keys are split by data disk.
> This is pretty serious, because all our nodes right now are consuming CPU doing this
for multiple tables, it seems.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org


Mime
View raw message