cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stefano Ortolani (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-12100) Compactions are stuck after TRUNCATE
Date Tue, 28 Jun 2016 08:51:57 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352614#comment-15352614
] 

Stefano Ortolani commented on CASSANDRA-12100:
----------------------------------------------

I see nothing weird in the logs but the following two things:

system.log

{noformat}
INFO  [SharedPool-Worker-2] 2016-06-27 22:12:06,323 Message.java:605 - Unexpected exception
during request; channel = [id: 0x6f0c2f56, /10.0.0.8:47592 :> /10.0.0.5:9042]
java.io.IOException: Error while read(...): Connection reset by peer
	at io.netty.channel.epoll.Native.readAddress(Native Method) ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
	at io.netty.channel.epoll.EpollSocketChannel$EpollSocketUnsafe.doReadBytes(EpollSocketChannel.java:675)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
	at io.netty.channel.epoll.EpollSocketChannel$EpollSocketUnsafe.epollInReady(EpollSocketChannel.java:714)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
	at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:326) ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:264) ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
INFO  [CompactionExecutor:3] 2016-06-27 22:45:47,213 AutoSavingCache.java:386 - Saved CounterCache
(1 items) in 169 ms
INFO  [CompactionExecutor:4] 2016-06-27 22:45:47,713 AutoSavingCache.java:386 - Saved KeyCache
(15158 items) in 671 ms
INFO  [IndexSummaryManager:1] 2016-06-27 22:45:58,397 IndexSummaryRedistribution.java:74 -
Redistributing index summaries
INFO  [IndexSummaryManager:1] 2016-06-27 23:45:58,440 IndexSummaryRedistribution.java:74 -
Redistributing index summaries
INFO  [SharedPool-Worker-17] 2016-06-28 00:33:57,301 Message.java:605 - Unexpected exception
during request; channel = [id: 0x52e03a04, /10.0.0.8:53218 :> /10.0.0.5:9042]
java.io.IOException: Error while read(...): Connection reset by peer
	at io.netty.channel.epoll.Native.readAddress(Native Method) ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
	at io.netty.channel.epoll.EpollSocketChannel$EpollSocketUnsafe.doReadBytes(EpollSocketChannel.java:675)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
	at io.netty.channel.epoll.EpollSocketChannel$EpollSocketUnsafe.epollInReady(EpollSocketChannel.java:714)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
	at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:326) ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:264) ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
~[netty-all-4.0.23.Final.jar:4.0.23.Final]
	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
INFO  [SharedPool-Worker-4] 2016-06-28 00:33:57,687 Message.java:605 - Unexpected exception
during request; channel = [id: 0x21eeb0b8, /10.0.0.8:53251 :> /10.0.0.5:9042]
{noformat}

debug.log

{noformat}
DEBUG [GossipStage:1] 2016-06-27 22:12:06,112 FailureDetector.java:456 - Ignoring interval
time of 2713433121 for /10.0.0.7
INFO  [SharedPool-Worker-2] 2016-06-27 22:12:06,323 Message.java:605 - Unexpected exception
during request; channel = [id: 0x6f0c2f56, /10.0.0.8:47592 :> /10.0.0.5:9042]
{noformat}

> Compactions are stuck after TRUNCATE
> ------------------------------------
>
>                 Key: CASSANDRA-12100
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-12100
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Stefano Ortolani
>
> Hi,
> since the upgrade to C* 3.0.7 I see compaction tasks getting stuck when truncating the
column family. I verified this on all nodes of the cluster.
> Pending compactions seem to disappear after restarting the node.
> {noformat}
> root@node10:~# nodetool -h localhost compactionstats
> pending tasks: 6
>                                      id   compaction type              keyspace     
          table   completed      total    unit   progress
>    24e1ad30-3cac-11e6-870d-5de740693258        Compaction   		     schema		         table_1
          0   57558382   bytes      0.00%
>    2be2e3b0-3cac-11e6-870d-5de740693258        Compaction   		     schema		         table_2
          0   65063705   bytes      0.00%
>    54de38f0-3cac-11e6-870d-5de740693258        Compaction   		     schema		         table_3
          0     187031   bytes      0.00%
>    31926ce0-3cac-11e6-870d-5de740693258        Compaction   		     schema		         table_4
          0   42951119   bytes      0.00%
>    3911ad00-3cac-11e6-870d-5de740693258        Compaction   		     schema		         table_5
          0   25918949   bytes      0.00%
>    3e6a8ab0-3cac-11e6-870d-5de740693258        Compaction   		     schema		         table_6
          0   65466210   bytes      0.00%
> Active compaction remaining time :   0h00m15s
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message