cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xihui He <xihu...@gmail.com>
Subject Infinite loop in SliceQueryFilter
Date Fri, 04 Dec 2015 16:43:59 GMT
Dear All,

Recently one of node in our cluster has high cpu load ~100%. It seems to me there is a infinite
loop in SliceQueryFilter.

The below log is repeated in 5000ms (range_request_timeout_in_ms).
TRACE [SharedPool-Worker-11] 2015-12-04 19:25:33,418 SliceQueryFilter.java:269 - collecting
1 of 2147483647: images:b608719e728d11e5812b57f4c5416142:false:62@1444838867382000

Our version is 2.19. Here is the bt:
org.apache.cassandra.db.composites.AbstractComposite.isEmpty(AbstractComposite.java:30)
org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:76)
org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:52)
org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:46)
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:83)
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:37)
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:82)
org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:173)
org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:156)
org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:146)
org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:125)
org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:99)
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:264)
org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:108)
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:82)
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:69)
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:314)
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:65)
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:2033)
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1876)
org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:357)
org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:85)
org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:47)
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64)
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
java.lang.Thread.run(Thread.java:745)

Appreciate if anyone could help.

Thanks in advance,
Xihui
Mime
View raw message