cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Petrov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-10756) Timeout failures in NativeTransportService.testConcurrentDestroys unit test
Date Wed, 13 Apr 2016 13:16:25 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-10756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15239212#comment-15239212
] 

Alex Petrov commented on CASSANDRA-10756:
-----------------------------------------

For sakes of experiment, I've changed the order (made a shutdown of {{workerGroup}} and {{eventExecutorGroup}}
before calling {{destroy}}), it seems unrelated, as there are different executors that are
being shut down. Also, there's an assert for {{allTerminated.get())}}, which confirms that
worker group and {{eventGroupExectuor}} are unrelated.

Same with {{emptyList}} replacement, since the call to {{Server::stop}} is synchronous, at
least one thread would succeed.

So it seems that it's some other group that's being shut down is causing this error. Maybe
roots are in the fact that the test times out...

> Timeout failures in NativeTransportService.testConcurrentDestroys unit test
> ---------------------------------------------------------------------------
>
>                 Key: CASSANDRA-10756
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10756
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Joel Knighton
>            Assignee: Alex Petrov
>
> History of test on trunk [here|http://cassci.datastax.com/job/trunk_testall/lastCompletedBuild/testReport/org.apache.cassandra.service/NativeTransportServiceTest/testConcurrentDestroys/history/].
> I've seen these failures across 3.0/trunk for a while. I ran the test looping locally
for a while and the timeout is fairly easy to reproduce. The timeout appears to be an indefinite
hang and not a timing issue.
> When the timeout occurs, the following stack trace is at the end of the logs for the
unit test.
> {code}
> ERROR [ForkJoinPool.commonPool-worker-1] 2015-11-22 21:30:53,635 Failed to submit a listener
notification task. Event loop shut down?
> java.util.concurrent.RejectedExecutionException: event executor terminated
> 	at io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:745)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> 	at io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:322)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> 	at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:728)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> 	at io.netty.util.concurrent.DefaultPromise.execute(DefaultPromise.java:671) [netty-all-4.0.23.Final.jar:4.0.23.Final]
> 	at io.netty.util.concurrent.DefaultPromise.notifyLateListener(DefaultPromise.java:641)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
> 	at io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:138) [netty-all-4.0.23.Final.jar:4.0.23.Final]
> 	at io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:93)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
> 	at io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:28)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
> 	at io.netty.channel.group.DefaultChannelGroupFuture.<init>(DefaultChannelGroupFuture.java:116)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
> 	at io.netty.channel.group.DefaultChannelGroup.close(DefaultChannelGroup.java:275) [netty-all-4.0.23.Final.jar:4.0.23.Final]
> 	at io.netty.channel.group.DefaultChannelGroup.close(DefaultChannelGroup.java:167) [netty-all-4.0.23.Final.jar:4.0.23.Final]
> 	at org.apache.cassandra.transport.Server$ConnectionTracker.closeAll(Server.java:277)
[main/:na]
> 	at org.apache.cassandra.transport.Server.close(Server.java:180) [main/:na]
> 	at org.apache.cassandra.transport.Server.stop(Server.java:116) [main/:na]
> 	at java.util.Collections$SingletonSet.forEach(Collections.java:4767) ~[na:1.8.0_60]
> 	at org.apache.cassandra.service.NativeTransportService.stop(NativeTransportService.java:136)
~[main/:na]
> 	at org.apache.cassandra.service.NativeTransportService.destroy(NativeTransportService.java:144)
~[main/:na]
> 	at org.apache.cassandra.service.NativeTransportServiceTest.lambda$withService$102(NativeTransportServiceTest.java:201)
~[classes/:na]
> 	at java.util.stream.IntPipeline$3$1.accept(IntPipeline.java:233) ~[na:1.8.0_60]
> 	at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110) ~[na:1.8.0_60]
> 	at java.util.Spliterator$OfInt.forEachRemaining(Spliterator.java:693) ~[na:1.8.0_60]
> 	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[na:1.8.0_60]
> 	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[na:1.8.0_60]
> 	at java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:747) ~[na:1.8.0_60]
> 	at java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:721) ~[na:1.8.0_60]
> 	at java.util.stream.AbstractTask.compute(AbstractTask.java:316) ~[na:1.8.0_60]
> 	at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731) ~[na:1.8.0_60]
> 	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) ~[na:1.8.0_60]
> 	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) ~[na:1.8.0_60]
> 	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) ~[na:1.8.0_60]
> 	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) ~[na:1.8.0_60]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message