asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wenhai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ASTERIXDB-2156) Encounter error during feed, in both inverted index and rtree
Date Wed, 08 Nov 2017 16:55:00 GMT

    [ https://issues.apache.org/jira/browse/ASTERIXDB-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244307#comment-16244307
] 

Wenhai commented on ASTERIXDB-2156:
-----------------------------------

I should be mentioned that, in the current master we will encounter the first error shown
above where the database will corrupt.on 300 million twitter datasets, regardless of the indexes
(We try to reduce the index with only only fullindex on twitter text, that error still exists.)
After we return the version in Oct. 22, the error disappears.

> Encounter error during feed, in both inverted index and rtree
> -------------------------------------------------------------
>
>                 Key: ASTERIXDB-2156
>                 URL: https://issues.apache.org/jira/browse/ASTERIXDB-2156
>             Project: Apache AsterixDB
>          Issue Type: Bug
>         Environment: 4nc 8 partitions per node. Linux
>            Reporter: Wenhai
>
> Hi, all
> When we try cloudberry recently, we encounter the following error when we use feed on
0.3 billions twitter records.
> We can reproduce the following errors
> {\noformat}
> SEVERE: Failed merge operation on {"class" : "LSMInvertedIndex", "dir" : "/kolla/asterixdb/asterix/node4/8/storage/storage/partition_31/twitter/ds_tweet_idx_text_idx",
"memory" : 2, "disk" : 14}
> java.lang.Error: Maximum lock count exceeded
>         at java.util.concurrent.locks.ReentrantReadWriteLock$Sync.fullTryAcquireShared(ReentrantReadWriteLock.java:528)
>         at java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryAcquireShared(ReentrantReadWriteLock.java:488)
>         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1282)
>         at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
>         at org.apache.hyracks.storage.common.buffercache.CachedPage.acquireReadLatch(CachedPage.java:120)
>         at org.apache.hyracks.storage.am.btree.impls.BTree.acquireLatch(BTree.java:542)
>         at org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:570)
>         at org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:596)
>         at org.apache.hyracks.storage.am.btree.impls.BTree.search(BTree.java:198)
>         at org.apache.hyracks.storage.am.btree.impls.BTree.access$300(BTree.java:69)
>         at org.apache.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.search(BTree.java:902)
>         at org.apache.hyracks.storage.am.lsm.invertedindex.ondisk.OnDiskInvertedIndexRangeSearchCursor.open(OnDiskInvertedIndexRangeSearchCursor.java:74)
>         at org.apache.hyracks.storage.am.lsm.invertedindex.ondisk.OnDiskInvertedIndex$OnDiskInvertedIndexAccessor.rangeSearch(OnDiskInvertedIndex.java:463)
>         at org.apache.hyracks.storage.am.lsm.invertedindex.impls.LSMInvertedIndexRangeSearchCursor.open(LSMInvertedIndexRangeSearchCursor.java:68)
>         at org.apache.hyracks.storage.am.lsm.invertedindex.impls.LSMInvertedIndex.search(LSMInvertedIndex.java:223)
>         at org.apache.hyracks.storage.am.lsm.invertedindex.impls.LSMInvertedIndex.doMerge(LSMInvertedIndex.java:353)
>         at org.apache.hyracks.storage.am.lsm.common.impls.AbstractLSMIndex.merge(AbstractLSMIndex.java:669)
>         at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.merge(LSMHarness.java:573)
>         at org.apache.hyracks.storage.am.lsm.invertedindex.impls.LSMInvertedIndexAccessor.merge(LSMInvertedIndexAccessor.java:124)
>         at org.apache.hyracks.storage.am.lsm.common.impls.MergeOperation.call(MergeOperation.java:45)
>         at org.apache.hyracks.storage.am.lsm.common.impls.MergeOperation.call(MergeOperation.java:30)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>         at java.lang.Thread.run(Thread.java:748)
> {\noformat}
> or, 
> {\noformat}
> Nov 07, 2017 3:39:55 PM org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness merge
> SEVERE: Failed merge operation on {"class" : "LSMRTreeWithAntiMatterTuples", "dir" :
"/kolla/asterixdb/asterix/node4/1/storage/storage/partition_24/twitter/ds_tweet_idx_location_idx",
"memory" : 2, "disk" : 19}
> org.apache.hyracks.algebricks.common.exceptions.NotImplementedException: Value provider
for type missing is not implemented
>         at org.apache.asterix.dataflow.data.nontagged.valueproviders.PrimitiveValueProviderFactory$1.getValue(PrimitiveValueProviderFactory.java:60)
>         at org.apache.hyracks.storage.am.rtree.frames.RTreeNSMFrame.calculateMBRImpl(RTreeNSMFrame.java:131)
>         at org.apache.hyracks.storage.am.rtree.frames.RTreeNSMFrame.adjustMBR(RTreeNSMFrame.java:152)
>         at org.apache.hyracks.storage.am.rtree.impls.RTree$RTreeBulkLoader.propagateBulk(RTree.java:1047)
>         at org.apache.hyracks.storage.am.rtree.impls.RTree$RTreeBulkLoader.add(RTree.java:948)
>         at org.apache.hyracks.storage.am.lsm.common.impls.LSMIndexBulkLoader.add(LSMIndexBulkLoader.java:54)
>         at org.apache.hyracks.storage.am.lsm.common.impls.ChainedLSMDiskComponentBulkLoader.add(ChainedLSMDiskComponentBulkLoader.java:54)
>         at org.apache.hyracks.storage.am.lsm.rtree.impls.LSMRTreeWithAntiMatterTuples.doMerge(LSMRTreeWithAntiMatterTuples.java:195)
>         at org.apache.hyracks.storage.am.lsm.common.impls.AbstractLSMIndex.merge(AbstractLSMIndex.java:667)
>         at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.merge(LSMHarness.java:574)
>         at org.apache.hyracks.storage.am.lsm.common.impls.LSMTreeIndexAccessor.merge(LSMTreeIndexAccessor.java:128)
>         at org.apache.hyracks.storage.am.lsm.common.impls.MergeOperation.call(MergeOperation.java:45)
>         at org.apache.hyracks.storage.am.lsm.common.impls.MergeOperation.call(MergeOperation.java:30)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>         at java.lang.Thread.run(Thread.java:748)
> {\noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message