cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jaydeepkumar Chovatia (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CASSANDRA-14674) Repair Validation message request could get stuck forever at sender side
Date Wed, 29 Aug 2018 16:54:00 GMT
Jaydeepkumar Chovatia created CASSANDRA-14674:
-------------------------------------------------

             Summary: Repair Validation message request could get stuck forever at sender
side
                 Key: CASSANDRA-14674
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-14674
             Project: Cassandra
          Issue Type: Bug
          Components: Repair
            Reporter: Jaydeepkumar Chovatia
            Assignee: Jaydeepkumar Chovatia


Validation request message as part of repair are currently sent as [sendOneWay|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/repair/ValidationTask.java#L56] and
then it waits at [Futures.getUnchecked|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/repair/RepairJob.java#L160]].
If sender doesn’t hear back from receiver for whatever reason then thread is blocked forever.
I’ve reproduced following stack trace at sender side by deliberately ignoring [VALIDATION_REQUST|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/repair/RepairMessageVerbHandler.java#L114]
at receiver side.
{quote}
"Repair#1:1" #301 daemon prio=5 os_prio=0 tid=0x00007f5a62060800 nid=0x13198 waiting on condition
[0x00007f5a5cc6c000]
 java.lang.Thread.State: WAITING (parking)
 at sun.misc.Unsafe.park(Native Method)
 parking to wait for <0x00000005c6ba9630> (a com.google.common.util.concurrent.AbstractFuture$Sync)
 at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
 at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
 at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
 at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
 at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:285)
 at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
 at com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:137)
 at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1509)
 at org.apache.cassandra.repair.RepairJob.run(RepairJob.java:160)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
 at org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$3/1858015030.run(Unknown Source)
 at java.lang.Thread.run(Thread.java:745)
{quote}

AFAIK we should be using {{sendRR}} for this instead of {{sendOneWay}}. Please let me know
if my understanding is correct or not.

I am working on a fix to make it {{sendRR}}.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org


Mime
View raw message