cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeremiah Jordan (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CASSANDRA-9882) DTCS (maybe other strategies) can block flushing when there are lots of sstables
Date Thu, 23 Jul 2015 19:04:05 GMT
Jeremiah Jordan created CASSANDRA-9882:
------------------------------------------

             Summary: DTCS (maybe other strategies) can block flushing when there are lots
of sstables
                 Key: CASSANDRA-9882
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9882
             Project: Cassandra
          Issue Type: Bug
          Components: Core
            Reporter: Jeremiah Jordan


MemtableFlushWriter tasks can get blocked by Compaction getNextBackgroundTask.  This is in
a wonky cluster with 200k sstables in the CF, but seems bad for flushing to be blocked by
getNextBackgroundTask when we are trying to make these new "smart" strategies that may take
some time to calculate what to do.

{noformat}
"MemtableFlushWriter:21" daemon prio=10 tid=0x00007ff7ad965000 nid=0x6693 waiting for monitor
entry [0x00007ff78a667000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at org.apache.cassandra.db.compaction.WrappingCompactionStrategy.handleNotification(WrappingCompactionStrategy.java:237)
	- waiting to lock <0x00000006fcdbbf60> (a org.apache.cassandra.db.compaction.WrappingCompactionStrategy)
	at org.apache.cassandra.db.DataTracker.notifyAdded(DataTracker.java:518)
	at org.apache.cassandra.db.DataTracker.replaceFlushed(DataTracker.java:178)
	at org.apache.cassandra.db.compaction.AbstractCompactionStrategy.replaceFlushed(AbstractCompactionStrategy.java:234)
	at org.apache.cassandra.db.ColumnFamilyStore.replaceFlushed(ColumnFamilyStore.java:1475)
	at org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:336)
	at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
	at org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1127)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)

   Locked ownable synchronizers:
	- <0x0000000743b3ac38> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"MemtableFlushWriter:19" daemon prio=10 tid=0x00007ff7ac57a000 nid=0x649b waiting for monitor
entry [0x00007ff78b8ee000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at org.apache.cassandra.db.compaction.WrappingCompactionStrategy.handleNotification(WrappingCompactionStrategy.java:237)
	- waiting to lock <0x00000006fcdbbf60> (a org.apache.cassandra.db.compaction.WrappingCompactionStrategy)
	at org.apache.cassandra.db.DataTracker.notifyAdded(DataTracker.java:518)
	at org.apache.cassandra.db.DataTracker.replaceFlushed(DataTracker.java:178)
	at org.apache.cassandra.db.compaction.AbstractCompactionStrategy.replaceFlushed(AbstractCompactionStrategy.java:234)
	at org.apache.cassandra.db.ColumnFamilyStore.replaceFlushed(ColumnFamilyStore.java:1475)
	at org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:336)
	at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
	at org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1127)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)

"CompactionExecutor:14" daemon prio=10 tid=0x00007ff7ad359800 nid=0x4d59 runnable [0x00007fecce3ea000]
   java.lang.Thread.State: RUNNABLE
	at org.apache.cassandra.io.sstable.SSTableReader.equals(SSTableReader.java:628)
	at com.google.common.collect.ImmutableSet.construct(ImmutableSet.java:206)
	at com.google.common.collect.ImmutableSet.construct(ImmutableSet.java:220)
	at com.google.common.collect.ImmutableSet.access$000(ImmutableSet.java:74)
	at com.google.common.collect.ImmutableSet$Builder.build(ImmutableSet.java:531)
	at com.google.common.collect.Sets$1.immutableCopy(Sets.java:606)
	at org.apache.cassandra.db.ColumnFamilyStore.getOverlappingSSTables(ColumnFamilyStore.java:1352)
	at org.apache.cassandra.db.compaction.DateTieredCompactionStrategy.getNextBackgroundSSTables(DateTieredCompactionStrategy.java:88)
	at org.apache.cassandra.db.compaction.DateTieredCompactionStrategy.getNextBackgroundTask(DateTieredCompactionStrategy.java:65)
	- locked <0x00000006fcdbbf00> (a org.apache.cassandra.db.compaction.DateTieredCompactionStrategy)
	at org.apache.cassandra.db.compaction.WrappingCompactionStrategy.getNextBackgroundTask(WrappingCompactionStrategy.java:72)
	- locked <0x00000006fcdbbf60> (a org.apache.cassandra.db.compaction.WrappingCompactionStrategy)
	at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:238)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
{noformat}





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message