cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eduard Tudenhoefner (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-11076) LEAK detected after bootstrapping a new node
Date Mon, 08 Feb 2016 16:36:40 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-11076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137182#comment-15137182
] 

Eduard Tudenhoefner commented on CASSANDRA-11076:
-------------------------------------------------

[~krummas] for a while, it was happening every time in our tests, but as of Feb 3rd I also
cannot reproduce the issue anymore in our tests. 

> LEAK detected after bootstrapping a new node
> --------------------------------------------
>
>                 Key: CASSANDRA-11076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-11076
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Eduard Tudenhoefner
>            Assignee: Marcus Eriksson
>             Fix For: 3.0.x
>
>
> Sequence of events:
> * start up a 2 node cluster
> * bootstrap one additional node so that the cluster consists of 3 nodes in total
> * the bootstrapped node will contain the LEAK error in the log file
> {code}
> INFO  [main] 2016-01-26 10:59:06,206  Server.java:162 - Starting listening for CQL clients
on /0.0.0.0:9042 (unencrypted)...
> INFO  [main] 2016-01-26 10:59:06,269  ThriftServer.java:119 - Binding thrift service
to /0.0.0.0:9160
> INFO  [Thread-6] 2016-01-26 10:59:06,280  ThriftServer.java:136 - Listening for thrift
clients...
> INFO  [HANDSHAKE-/10.200.178.183] 2016-01-26 10:59:06,703  OutboundTcpConnection.java:503
- Handshaking version with /10.200.178.183
> INFO  [RMI TCP Connection(4)-10.200.178.183] 2016-01-26 10:59:20,079  StorageService.java:1099
- rebuild from dc: (any dc)
> INFO  [RMI TCP Connection(4)-10.200.178.183] 2016-01-26 10:59:20,090  RangeStreamer.java:339
- Some ranges of [(-9223372036854775808,-3074457345618258603], (-3074457345618258603,3074457345618258602]]
are already available. Skipping streaming those ranges.
> INFO  [RMI TCP Connection(4)-10.200.178.183] 2016-01-26 10:59:20,091  RangeStreamer.java:339
- Some ranges of [(-9223372036854775808,-3074457345618258603], (-3074457345618258603,3074457345618258602]]
are already available. Skipping streaming those ranges.
> INFO  [RMI TCP Connection(4)-10.200.178.183] 2016-01-26 10:59:20,092  RangeStreamer.java:339
- Some ranges of [(3074457345618258602,-9223372036854775808], (-9223372036854775808,-3074457345618258603],
(-3074457345618258603,3074457345618258602]] are already available. Skipping streaming those
ranges.
> INFO  [RMI TCP Connection(4)-10.200.178.183] 2016-01-26 10:59:20,093  RangeStreamer.java:339
- Some ranges of [(3074457345618258602,-9223372036854775808], (-9223372036854775808,-3074457345618258603],
(-3074457345618258603,3074457345618258602]] are already available. Skipping streaming those
ranges.
> INFO  [RMI TCP Connection(4)-10.200.178.183] 2016-01-26 10:59:20,094  StreamResultFuture.java:86
- [Stream #d9bbe900-c41b-11e5-8540-d1e65b596c03] Executing streaming plan for Rebuild
> INFO  [StreamConnectionEstablisher:3] 2016-01-26 10:59:20,095  StreamSession.java:238
- [Stream #d9bbe900-c41b-11e5-8540-d1e65b596c03] Starting streaming to /10.200.178.185
> INFO  [StreamConnectionEstablisher:4] 2016-01-26 10:59:20,096  StreamSession.java:238
- [Stream #d9bbe900-c41b-11e5-8540-d1e65b596c03] Starting streaming to /10.200.178.193
> INFO  [StreamConnectionEstablisher:4] 2016-01-26 10:59:20,097  StreamCoordinator.java:213
- [Stream #d9bbe900-c41b-11e5-8540-d1e65b596c03, ID#0] Beginning stream session with /10.200.178.193
> INFO  [StreamConnectionEstablisher:3] 2016-01-26 10:59:20,098  StreamCoordinator.java:213
- [Stream #d9bbe900-c41b-11e5-8540-d1e65b596c03, ID#0] Beginning stream session with /10.200.178.185
> INFO  [STREAM-IN-/10.200.178.185] 2016-01-26 10:59:20,102  StreamResultFuture.java:182
- [Stream #d9bbe900-c41b-11e5-8540-d1e65b596c03] Session with /10.200.178.185 is complete
> INFO  [STREAM-IN-/10.200.178.193] 2016-01-26 10:59:20,359  StreamResultFuture.java:182
- [Stream #d9bbe900-c41b-11e5-8540-d1e65b596c03] Session with /10.200.178.193 is complete
> INFO  [STREAM-IN-/10.200.178.193] 2016-01-26 10:59:20,362  StreamResultFuture.java:214
- [Stream #d9bbe900-c41b-11e5-8540-d1e65b596c03] All sessions completed
> ERROR [Reference-Reaper:1] 2016-01-26 11:00:39,410  Ref.java:197 - LEAK DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@1c7d1dcf) to @2011417651 was not released
before the reference was garbage collected
> ERROR [Reference-Reaper:1] 2016-01-26 11:00:39,411  Ref.java:228 - Allocate trace org.apache.cassandra.utils.concurrent.Ref$State@1c7d1dcf:
> Thread[SharedPool-Worker-6,5,main]
> 	at java.lang.Thread.getStackTrace(Thread.java:1552)
> 	at org.apache.cassandra.utils.concurrent.Ref$Debug.<init>(Ref.java:218)
> 	at org.apache.cassandra.utils.concurrent.Ref$State.<init>(Ref.java:148)
> 	at org.apache.cassandra.utils.concurrent.Ref.<init>(Ref.java:70)
> 	at org.apache.cassandra.utils.memory.BufferPool$Chunk.setAttachment(BufferPool.java:646)
> 	at org.apache.cassandra.utils.memory.BufferPool$Chunk.get(BufferPool.java:786)
> 	at org.apache.cassandra.utils.memory.BufferPool$Chunk.get(BufferPool.java:776)
> 	at org.apache.cassandra.utils.memory.BufferPool$LocalPool.get(BufferPool.java:379)
> 	at org.apache.cassandra.utils.memory.BufferPool.maybeTakeFromPool(BufferPool.java:139)
> 	at org.apache.cassandra.utils.memory.BufferPool.takeFromPool(BufferPool.java:113)
> 	at org.apache.cassandra.utils.memory.BufferPool.get(BufferPool.java:92)
> 	at org.apache.cassandra.io.util.RandomAccessReader.allocateBuffer(RandomAccessReader.java:87)
> 	at org.apache.cassandra.io.compress.CompressedRandomAccessReader.access$100(CompressedRandomAccessReader.java:38)
> 	at org.apache.cassandra.io.compress.CompressedRandomAccessReader$Builder.createBuffer(CompressedRandomAccessReader.java:275)
> 	at org.apache.cassandra.io.util.RandomAccessReader.<init>(RandomAccessReader.java:74)
> 	at org.apache.cassandra.io.compress.CompressedRandomAccessReader.<init>(CompressedRandomAccessReader.java:59)
> 	at org.apache.cassandra.io.compress.CompressedRandomAccessReader$Builder.build(CompressedRandomAccessReader.java:283)
> 	at org.apache.cassandra.io.util.CompressedSegmentedFile.createReader(CompressedSegmentedFile.java:145)
> 	at org.apache.cassandra.io.util.SegmentedFile.createReader(SegmentedFile.java:133)
> 	at org.apache.cassandra.io.sstable.format.SSTableReader.getFileDataInput(SSTableReader.java:1762)
> 	at org.apache.cassandra.db.columniterator.AbstractSSTableIterator.<init>(AbstractSSTableIterator.java:93)
> 	at org.apache.cassandra.db.columniterator.SSTableIterator.<init>(SSTableIterator.java:46)
> 	at org.apache.cassandra.db.columniterator.SSTableIterator.<init>(SSTableIterator.java:36)
> 	at org.apache.cassandra.io.sstable.format.big.BigTableReader.iterator(BigTableReader.java:62)
> 	at org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDiskInternal(SinglePartitionReadCommand.java:547)
> 	at org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDisk(SinglePartitionReadCommand.java:459)
> 	at org.apache.cassandra.db.SinglePartitionReadCommand.queryStorage(SinglePartitionReadCommand.java:325)
> 	at org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:350)
> 	at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1719)
> 	at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2375)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> 	at org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
> 	at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> 	at java.lang.Thread.run(Thread.java:745)
> INFO  [CompactionExecutor:5] 2016-01-26 11:05:49,416  CompactionManager.java:1456 - Compaction
interrupted: Compaction@a17c09d1-c41b-11e5-8d26-7dd65babdc03(keyspace1, standard1, 0/229706280)bytes
> INFO  [CompactionExecutor:5] 2016-01-26 11:06:07,332  CompactionManager.java:1456 - Compaction
interrupted: Compaction@a17c09d1-c41b-11e5-8d26-7dd65babdc03(keyspace1, standard1, 86299/211434370)bytes
> INFO  [CompactionExecutor:4] 2016-01-26 11:07:31,662  CompactionManager.java:1456 - Compaction
interrupted: Compaction@a17c09d1-c41b-11e5-8d26-7dd65babdc03(keyspace1, standard1, 118545029/211434370)bytes
> {code}
> This happened with *Cassandra version: 3.0.1.816*



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message