Return-Path: X-Original-To: apmail-cassandra-commits-archive@www.apache.org Delivered-To: apmail-cassandra-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 20D4E18EA3 for ; Mon, 14 Sep 2015 13:53:46 +0000 (UTC) Received: (qmail 84503 invoked by uid 500); 14 Sep 2015 13:53:46 -0000 Delivered-To: apmail-cassandra-commits-archive@cassandra.apache.org Received: (qmail 84464 invoked by uid 500); 14 Sep 2015 13:53:46 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 84446 invoked by uid 99); 14 Sep 2015 13:53:45 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 14 Sep 2015 13:53:45 +0000 Date: Mon, 14 Sep 2015 13:53:45 +0000 (UTC) From: "Adam Bliss (JIRA)" To: commits@cassandra.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (CASSANDRA-10270) Cassandra stops compacting MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CASSANDRA-10270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14743535#comment-14743535 ] Adam Bliss commented on CASSANDRA-10270: ---------------------------------------- Quick update: I have one table on STCS which has been working great with the workaround posted. However, I have another copy which I tried altering to LeveledCompactionStrategy and it seems to still be exhibiting the same behavior (compaction eventually quits with ~400 sstables on disk). > Cassandra stops compacting > -------------------------- > > Key: CASSANDRA-10270 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10270 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: linux (google cloud click-to-deploy, default settings, 3 nodes) > Reporter: Adam Bliss > Assignee: Marcus Eriksson > Priority: Critical > Fix For: 3.0.0 rc1, 2.2.2 > > Attachments: system.log.gz, system.txt.gz > > > My cassandra cluster won't keep compacting. I notice that if I restart, it does compact for a while, but after a time it stops. As a result, after adding a bunch of rows, I ended up with about 1000 sstables per node. > I'll attach more logs in a minute, but it seems like this might be the most relevant part: > {noformat} > INFO [CompactionExecutor:1] 2015-09-04 14:22:55,796 CompactionManager.java:1433 - Compaction interrupted: Compaction@fff9bcd0-3b1f-11e5-8df6-33158d7bf3bf(megacrawl2, ranks_by_domain, 812501702/7543091905)bytes > DEBUG [CompactionExecutor:1] 2015-09-04 14:22:55,797 CompactionManager.java:1437 - Full interruption stack trace: > org.apache.cassandra.db.compaction.CompactionInterruptedException: Compaction interrupted: Compaction@fff9bcd0-3b1f-11e5-8df6-33158d7bf3bf(megacrawl2, ranks_by_ > domain, 812501702/7543091905)bytes > at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:180) ~[apache-cassandra-2.2.1.jar:2.2.1] > at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[apache-cassandra-2.2.1.jar:2.2.1] > at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:74) ~[apache-cassandra-2.2.1.jar:2.2.1] > at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) ~[apache-cassandra-2.2.1.jar:2.2.1] at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:236) ~[apache-cassandra-2.2.1.jar:2.2.1] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_79] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_79] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_79] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_79] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] > DEBUG [CompactionExecutor:1] 2015-09-04 14:22:55,797 CompactionManager.java:222 - Checking system.local DEBUG [CompactionExecutor:1] 2015-09-04 14:22:55,797 SizeTieredCompactionStrategy.java:85 - Compaction buckets are [[BigTableReader(path='/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/la-83-big-Data.db'), BigTableReader(path='/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/la-81-big-Data.db'), > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)