Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Sun, 23 Apr 2017 13:48:04 +0000 (UTC)
From: "Tom van der Woerdt (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.13066175.1492954005000.28551.1492955284221@Atlassian.JIRA>
In-Reply-To: <JIRA.13066175.1492954005000@Atlassian.JIRA>
References: <JIRA.13066175.1492954005000@Atlassian.JIRA> <JIRA.13066175.1492954005965@jira-lw-us.apache.org>
Subject: [jira] [Resolved] (CASSANDRA-13469) Compaction stops; new sstables
 aren't recognized
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Sun, 23 Apr 2017 13:48:10 -0000


     [ https://issues.apache.org/jira/browse/CASSANDRA-13469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tom van der Woerdt resolved CASSANDRA-13469.
--------------------------------------------
    Resolution: Invalid

... never mind. This is most likely a configuration issue on my side.

> Compaction stops; new sstables aren't recognized
> ------------------------------------------------
>
>                 Key: CASSANDRA-13469
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-13469
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Tom van der Woerdt
>
> cfstats shows me this, so I know I'm in trouble :
> {code}
>                 SSTable count: 3419
>                 SSTables in each level: [2, 20/10, 154/100, 1034/1000, 0, 0, 0, 0, 0]
> {code}
> Disk space on these nodes (about five of them) doubled overnight, due to the high write volume not getting compacted. `nodetool compactionstats' shows me zero pending compactions.
> After a restart things are "better" :
> {noformat}
>                 SSTable count: 3432
>                 SSTables in each level: [1280/4, 21/10, 262/100, 1851/1000, 15, 0, 0, 0, 0]
> {noformat}
> compactionstats also shows some compactions again.
> jstack for one compaction thread (there are 24 of them, all looking the exact same) :
> {noformat}
> "CompactionExecutor:73017" #203194 daemon prio=1 os_prio=4 tid=0x00007edd1e597c00 nid=0xf7e4 waiting on condition [0x00007efd16782000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
>     at sun.misc.Unsafe.park(Native Method)
>     - parking to wait for  <0x00000002c1859520> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>     at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
>     at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
>     at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
>     at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
>     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>     at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
>     at org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$3/335359181.run(Unknown Source)
>     at java.lang.Thread.run(Thread.java:745)
> {noformat}
> Sadly I'm pretty clueless about what happened here. :(
> Oh, and tpstats :
> {noformat}
> Pool Name                    Active   Pending      Completed   Blocked  All time blocked
> MutationStage                     0         0     1098466484         0                 0
> ViewMutationStage                 0         0              0         0                 0
> ReadStage                         2         0      759723051         0                 0
> RequestResponseStage              0         0     1773795694         0                 0
> ReadRepairStage                   0         0       37372223         0                 0
> CounterMutationStage              0         0              0         0                 0
> MiscStage                         0         0              0         0                 0
> CompactionExecutor                0         0         336794         0                 0
> MemtableReclaimMemory             0         0          75324         0                 0
> PendingRangeCalculator            0         0             68         0                 0
> GossipStage                       0         0        1363346         0                 0
> SecondaryIndexManagement          0         0              0         0                 0
> HintsDispatcher                   1         1             38         0                 0
> MigrationStage                    0         0         274693         0                 0
> MemtablePostFlush                 0         0         248358         0                 0
> ValidationExecutor                0         0           7882         0                 0
> Sampler                           0         0              0         0                 0
> MemtableFlushWriter               0         0          75324         0                 0
> InternalResponseStage             0         0         259871         0                 0
> AntiEntropyStage                  0         0          20469         0                 0
> CacheCleanupExecutor              0         0              0         0                 0
> Native-Transport-Requests         0         0     1489435995         0            188985
> Message type           Dropped
> READ                         2
> RANGE_SLICE                  0
> _TRACE                       0
> HINT                         0
> MUTATION                    20
> COUNTER_MUTATION             0
> BATCH_STORE                  0
> BATCH_REMOVE                 0
> REQUEST_RESPONSE             0
> PAGED_RANGE                  0
> READ_REPAIR                  4
> {noformat}


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)