asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jianfeng Jia (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ASTERIXDB-1641) Hit HYR002 exception when query a large dataset
Date Mon, 12 Sep 2016 17:37:20 GMT

    [ https://issues.apache.org/jira/browse/ASTERIXDB-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15484727#comment-15484727
] 

Jianfeng Jia commented on ASTERIXDB-1641:
-----------------------------------------

It's a synchronous query. It's not trivial to change the client logic to use the asynchronous
one.  
As for the `sweepthreshold`, do you mean this property in the configuration file? 
{code}
  <property>
    <name>txn.lock.timeout.sweepthreshold</name>
    <value>10000</value>
    <description>The time in milliseconds the timeout thread waits between
      sweeps to detect timed-out transactions. (Default = "10000" // 10s)                
                                                                                      
    </description>
  </property>
{code}

Even that is the case, why it happens no matter how small my time range was set? 

> Hit HYR002 exception when query a large dataset
> -----------------------------------------------
>
>                 Key: ASTERIXDB-1641
>                 URL: https://issues.apache.org/jira/browse/ASTERIXDB-1641
>             Project: Apache AsterixDB
>          Issue Type: Bug
>          Components: Storage
>         Environment: Master
>            Reporter: Jianfeng Jia
>            Assignee: Ian Maxon
>            Priority: Blocker
>
> The twitter dataset now is above 100G. I hit an error when send a simple query to the
dataset: 
> {code}
> SEVERE: Job failed on account of:
> HYR0002: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException:
null
> org.apache.hyracks.api.exceptions.HyracksException: Job failed on account of:
> HYR0002: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException:
null
>         at org.apache.hyracks.control.cc.job.JobRun.waitForCompletion(JobRun.java:212)
>         at org.apache.hyracks.control.cc.work.WaitForJobCompletionWork$1.run(WaitForJobCompletionWork.java:48)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:744) 
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: HYR0002: org.apache.hyracks.api.exceptions.HyracksDataException:
org.apache.hyracks.api.exceptions.HyracksDat
> aException: null
>         at org.apache.hyracks.control.common.utils.ExceptionUtils.setNodeIds(ExceptionUtils.java:62)
>         at org.apache.hyracks.control.nc.Task.run(Task.java:319)
>         ... 3 more
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException:
org.apache.hyracks.api.exceptions.HyracksDataExceptio
> n: null
>         at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:218)
>         at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.initialize(SuperActivityOperatorNodePushable.java:83)
>         at org.apache.hyracks.control.nc.Task.run(Task.java:263)
>         ... 3 more
> Caused by: java.util.concurrent.ExecutionException: org.apache.hyracks.api.exceptions.HyracksDataException:
org.apache.hyracks.api.exceptions.HyracksDataException: null
>         at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>         at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:212)
>         ... 5 more
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException:
null
>         at org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.nextFrame(IndexSearchOperatorNodePushable.java:187)
>         at org.apache.hyracks.dataflow.common.comm.io.AbstractFrameAppender.write(AbstractFrameAppender.java:92)
>         at org.apache.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.flushAndReset(AbstractOneInputOneOutputOneFramePushRuntime.java:63)
>         at org.apache.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.flushIfNotFailed(AbstractOneInputOneOutputOneFramePushRuntime.java:
> 69)
>         at org.apache.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.close(AbstractOneInputOneOutputOneFramePushRuntime.java:55)
>         at org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.close(AssignRuntimeFactory.java:125)
>         at org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.close(AlgebricksMetaOperatorDescriptor.java:153)
>         at org.apache.hyracks.dataflow.std.sort.AbstractExternalSortRunMerger.process(AbstractExternalSortRunMerger.java:167)
>         at org.apache.hyracks.dataflow.std.sort.AbstractSorterOperatorDescriptor$MergeActivity$1.initialize(AbstractSorterOperatorDescriptor.java:194)
>         at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$initialize$0(SuperActivityOperatorNodePushable.java:83)
>         at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$$Lambda$1/1498483095.runAction(Unknown
Source)
>         at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$1.call(SuperActivityOperatorNodePushable.java:205)
>         at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$1.call(SuperActivityOperatorNodePushable.java:202)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>         ... 3 more
> Caused by: org.apache.hyracks.storage.am.btree.exceptions.BTreeException: org.apache.hyracks.api.exceptions.HyracksDataException:
null
>         at org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:827)
>         at org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:673)
>         at org.apache.hyracks.storage.am.btree.impls.BTree.search(BTree.java:204)
>         at org.apache.hyracks.storage.am.btree.impls.BTree.access$600(BTree.java:74)
>         at org.apache.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.search(BTree.java:966)
>         at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTreePointSearchCursor.hasNext(LSMBTreePointSearchCursor.java:74)
>         at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTreeSearchCursor.hasNext(LSMBTreeSearchCursor.java:86)
>         at org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.writeSearchResults(IndexSearchOperatorNodePushable.java:149)
>         at org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.nextFrame(IndexSearchOperatorNodePushable.java:184)
>         ... 16 more
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: null
>         at org.apache.hyracks.control.nc.io.IOManager.syncRead(IOManager.java:175)
>         at org.apache.hyracks.storage.common.buffercache.BufferCache.read(BufferCache.java:535)
>         at org.apache.hyracks.storage.common.buffercache.BufferCache.pin(BufferCache.java:211)
>         at org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:642)
>         at org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:673)
>         ... 24 more
> Caused by: java.nio.channels.ClosedChannelException
>         at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:94)
>         at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:673)
>         at org.apache.hyracks.control.nc.io.IOManager.syncRead(IOManager.java:163)
>         ... 28 more
> {code}
> [~imaxon] you could test it on actinium machine with the following query
> {code}
> count(for $t in dataset twitter.ds_tweet
> where $t.'create_at' >= datetime('2016-09-07T23:09:10.000Z') and $t.'create_at' <
datetime('2016-09-09T23:09:10.000Z') 
>          
> return $t)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message