asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jianfeng Jia (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (ASTERIXDB-1912) ClosedChannelException throws after canceling several queries
Date Wed, 17 May 2017 16:41:04 GMT

     [ https://issues.apache.org/jira/browse/ASTERIXDB-1912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jianfeng Jia updated ASTERIXDB-1912:
------------------------------------
    Description: 
It seems the canceling task closed some resource that shouldn't be closed. 

Here is the sequence to reproduce the issue
{code}
host="http://localhost:19002"
cat <<-'EOF' | curl -XPOST -d @- $host/query/service &
client_context_id=1&
statement=
select coll_count(
    (select value c from (select value t
    from twitter.ds_tweet t
    where t.`create_at` >= datetime('2017-01-01T12:16:00.000Z') and t.`create_at` <
datetime('2017-01-12T13:16:00.000Z') and ftcontains(t.`text`, ['rain'], {'mode':'all'})) as
c)
    ) as `count`;
EOF

sleep 1

#t.`create_at` >= datetime('2016-11-27T08:00:00.000Z') and t.`create_at` < datetime('2016-11-28T08:00:00.000Z')
and
var=`curl -v -XDELETE $host/admin/requests/running?client_context_id=1`
echo $var 

cat <<-'EOF' | curl -XPOST -d @- $host/query/service
client_context_id=2&
statement=
select coll_count(
    (select value c from (select value t
    from twitter.ds_tweet t
    where t.`create_at` >= datetime('2016-12-12T12:16:00.000Z') and t.`create_at` <
datetime('2016-12-12T13:16:00.000Z') and ftcontains(t.`text`, ['rain'], {'mode':'all'})) as
c)
    ) as `count`;
EOF
{code}

The first command sent a query, the second one canceled it after 1 second. These two worked
fine. 
Then if we submit the third query, the following exception will be thrown:

{code}
SEVERE: java.util.concurrent.ExecutionException: org.apache.hyracks.api.exceptions.HyracksDataException:
org.apache.hyracks.api.exceptions.HyracksDataException: java.nio.channels.ClosedChannelException
org.apache.hyracks.api.exceptions.HyracksDataException: java.util.concurrent.ExecutionException:
org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException:
java.nio.channels.ClosedChannelException
    at org.apache.hyracks.api.exceptions.HyracksDataException.create(HyracksDataException.java:58)
    at org.apache.hyracks.control.common.utils.ExceptionUtils.setNodeIds(ExceptionUtils.java:62)
    at org.apache.hyracks.control.nc.Task.run(Task.java:330)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:744)
Caused by: java.util.concurrent.ExecutionException: org.apache.hyracks.api.exceptions.HyracksDataException:
org.apache.hyracks.api.exceptions.HyracksDataException: java.nio.channels.ClosedChannelException
    at java.util.concurrent.FutureTask.report(FutureTask.java:122)
    at java.util.concurrent.FutureTask.get(FutureTask.java:192)
    at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:210)
    at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.initialize(SuperActivityOperatorNodePushable.java:80)
    at org.apache.hyracks.control.nc.Task.run(Task.java:273)
    ... 3 more 
Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException:
java.nio.channels.ClosedChannelException
    at org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.nextFrame(IndexSearchOperatorNodePushable.java:194)
    at org.apache.hyracks.dataflow.common.comm.io.AbstractFrameAppender.write(AbstractFrameAppender.java:92)
    at org.apache.hyracks.dataflow.common.comm.util.FrameUtils.appendToWriter(FrameUtils.java:117)
    at org.apache.hyracks.dataflow.std.sort.AbstractFrameSorter.flush(AbstractFrameSorter.java:172)
    at org.apache.hyracks.dataflow.std.sort.AbstractExternalSortRunMerger.process(AbstractExternalSortRunMerger.java:90)
    at org.apache.hyracks.dataflow.std.sort.AbstractSorterOperatorDescriptor$MergeActivity$1.initialize(AbstractSorterOperatorDescriptor.java:181)
    at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$initialize$1(SuperActivityOperatorNodePushable.java:80)
    at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$$Lambda$51/1152642867.run(Unknown
Source)
    at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$runInParallel$3(SuperActivityOperatorNodePushable.java:202)
    at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$$Lambda$52/69643057.call(Unknown
Source)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    ... 3 more 
Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: java.nio.channels.ClosedChannelException
    at org.apache.hyracks.control.nc.io.IOManager.syncRead(IOManager.java:209)
    at org.apache.hyracks.storage.common.buffercache.BufferCache.read(BufferCache.java:537)
    at org.apache.hyracks.storage.common.buffercache.BufferCache.pin(BufferCache.java:213)
    at org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:562)
    at org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:593)
    at org.apache.hyracks.storage.am.btree.impls.BTree.search(BTree.java:199)
    at org.apache.hyracks.storage.am.btree.impls.BTree.access$200(BTree.java:68)
    at org.apache.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.search(BTree.java:883)
    at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTreePointSearchCursor.hasNext(LSMBTreePointSearchCursor.java:71)
    at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTreeSearchCursor.hasNext(LSMBTreeSearchCursor.java:59)
    at org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.writeSearchResults(IndexSearchOperatorNodePushable.java:155)
    at org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.nextFrame(IndexSearchOperatorNodePushable.java:191)
    ... 13 more
Caused by: java.nio.channels.ClosedChannelException
    at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:94)
    at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:673)
    at org.apache.hyracks.control.nc.io.IOManager.syncRead(IOManager.java:197)
    ... 24 more
{code}

  was:
will fill a sequence query later 

{code}
SEVERE: java.util.concurrent.ExecutionException: org.apache.hyracks.api.exceptions.HyracksDataException:
org.apache.hyracks.api.exceptions.HyracksDataException: java.nio.channels.ClosedChannelException
org.apache.hyracks.api.exceptions.HyracksDataException: java.util.concurrent.ExecutionException:
org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException:
java.nio.channels.ClosedChannelException
    at org.apache.hyracks.api.exceptions.HyracksDataException.create(HyracksDataException.java:58)
    at org.apache.hyracks.control.common.utils.ExceptionUtils.setNodeIds(ExceptionUtils.java:62)
    at org.apache.hyracks.control.nc.Task.run(Task.java:330)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:744)
Caused by: java.util.concurrent.ExecutionException: org.apache.hyracks.api.exceptions.HyracksDataException:
org.apache.hyracks.api.exceptions.HyracksDataException: java.nio.channels.ClosedChannelException
    at java.util.concurrent.FutureTask.report(FutureTask.java:122)
    at java.util.concurrent.FutureTask.get(FutureTask.java:192)
    at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:210)
    at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.initialize(SuperActivityOperatorNodePushable.java:80)
    at org.apache.hyracks.control.nc.Task.run(Task.java:273)
    ... 3 more 
Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException:
java.nio.channels.ClosedChannelException
    at org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.nextFrame(IndexSearchOperatorNodePushable.java:194)
    at org.apache.hyracks.dataflow.common.comm.io.AbstractFrameAppender.write(AbstractFrameAppender.java:92)
    at org.apache.hyracks.dataflow.common.comm.util.FrameUtils.appendToWriter(FrameUtils.java:117)
    at org.apache.hyracks.dataflow.std.sort.AbstractFrameSorter.flush(AbstractFrameSorter.java:172)
    at org.apache.hyracks.dataflow.std.sort.AbstractExternalSortRunMerger.process(AbstractExternalSortRunMerger.java:90)
    at org.apache.hyracks.dataflow.std.sort.AbstractSorterOperatorDescriptor$MergeActivity$1.initialize(AbstractSorterOperatorDescriptor.java:181)
    at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$initialize$1(SuperActivityOperatorNodePushable.java:80)
    at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$$Lambda$51/1152642867.run(Unknown
Source)
    at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$runInParallel$3(SuperActivityOperatorNodePushable.java:202)
    at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$$Lambda$52/69643057.call(Unknown
Source)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    ... 3 more 
Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: java.nio.channels.ClosedChannelException
    at org.apache.hyracks.control.nc.io.IOManager.syncRead(IOManager.java:209)
    at org.apache.hyracks.storage.common.buffercache.BufferCache.read(BufferCache.java:537)
    at org.apache.hyracks.storage.common.buffercache.BufferCache.pin(BufferCache.java:213)
    at org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:562)
    at org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:593)
    at org.apache.hyracks.storage.am.btree.impls.BTree.search(BTree.java:199)
    at org.apache.hyracks.storage.am.btree.impls.BTree.access$200(BTree.java:68)
    at org.apache.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.search(BTree.java:883)
    at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTreePointSearchCursor.hasNext(LSMBTreePointSearchCursor.java:71)
    at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTreeSearchCursor.hasNext(LSMBTreeSearchCursor.java:59)
    at org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.writeSearchResults(IndexSearchOperatorNodePushable.java:155)
    at org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.nextFrame(IndexSearchOperatorNodePushable.java:191)
    ... 13 more
Caused by: java.nio.channels.ClosedChannelException
    at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:94)
    at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:673)
    at org.apache.hyracks.control.nc.io.IOManager.syncRead(IOManager.java:197)
    ... 24 more
{code}


> ClosedChannelException throws after canceling several queries 
> --------------------------------------------------------------
>
>                 Key: ASTERIXDB-1912
>                 URL: https://issues.apache.org/jira/browse/ASTERIXDB-1912
>             Project: Apache AsterixDB
>          Issue Type: Bug
>         Environment: master
>            Reporter: Jianfeng Jia
>
> It seems the canceling task closed some resource that shouldn't be closed. 
> Here is the sequence to reproduce the issue
> {code}
> host="http://localhost:19002"
> cat <<-'EOF' | curl -XPOST -d @- $host/query/service &
> client_context_id=1&
> statement=
> select coll_count(
>     (select value c from (select value t
>     from twitter.ds_tweet t
>     where t.`create_at` >= datetime('2017-01-01T12:16:00.000Z') and t.`create_at`
< datetime('2017-01-12T13:16:00.000Z') and ftcontains(t.`text`, ['rain'], {'mode':'all'}))
as c)
>     ) as `count`;
> EOF
> sleep 1
> #t.`create_at` >= datetime('2016-11-27T08:00:00.000Z') and t.`create_at` < datetime('2016-11-28T08:00:00.000Z')
and
> var=`curl -v -XDELETE $host/admin/requests/running?client_context_id=1`
> echo $var 
> cat <<-'EOF' | curl -XPOST -d @- $host/query/service
> client_context_id=2&
> statement=
> select coll_count(
>     (select value c from (select value t
>     from twitter.ds_tweet t
>     where t.`create_at` >= datetime('2016-12-12T12:16:00.000Z') and t.`create_at`
< datetime('2016-12-12T13:16:00.000Z') and ftcontains(t.`text`, ['rain'], {'mode':'all'}))
as c)
>     ) as `count`;
> EOF
> {code}
> The first command sent a query, the second one canceled it after 1 second. These two
worked fine. 
> Then if we submit the third query, the following exception will be thrown:
> {code}
> SEVERE: java.util.concurrent.ExecutionException: org.apache.hyracks.api.exceptions.HyracksDataException:
org.apache.hyracks.api.exceptions.HyracksDataException: java.nio.channels.ClosedChannelException
> org.apache.hyracks.api.exceptions.HyracksDataException: java.util.concurrent.ExecutionException:
org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException:
java.nio.channels.ClosedChannelException
>     at org.apache.hyracks.api.exceptions.HyracksDataException.create(HyracksDataException.java:58)
>     at org.apache.hyracks.control.common.utils.ExceptionUtils.setNodeIds(ExceptionUtils.java:62)
>     at org.apache.hyracks.control.nc.Task.run(Task.java:330)
>     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>     at java.lang.Thread.run(Thread.java:744)
> Caused by: java.util.concurrent.ExecutionException: org.apache.hyracks.api.exceptions.HyracksDataException:
org.apache.hyracks.api.exceptions.HyracksDataException: java.nio.channels.ClosedChannelException
>     at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>     at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>     at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:210)
>     at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.initialize(SuperActivityOperatorNodePushable.java:80)
>     at org.apache.hyracks.control.nc.Task.run(Task.java:273)
>     ... 3 more 
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException:
java.nio.channels.ClosedChannelException
>     at org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.nextFrame(IndexSearchOperatorNodePushable.java:194)
>     at org.apache.hyracks.dataflow.common.comm.io.AbstractFrameAppender.write(AbstractFrameAppender.java:92)
>     at org.apache.hyracks.dataflow.common.comm.util.FrameUtils.appendToWriter(FrameUtils.java:117)
>     at org.apache.hyracks.dataflow.std.sort.AbstractFrameSorter.flush(AbstractFrameSorter.java:172)
>     at org.apache.hyracks.dataflow.std.sort.AbstractExternalSortRunMerger.process(AbstractExternalSortRunMerger.java:90)
>     at org.apache.hyracks.dataflow.std.sort.AbstractSorterOperatorDescriptor$MergeActivity$1.initialize(AbstractSorterOperatorDescriptor.java:181)
>     at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$initialize$1(SuperActivityOperatorNodePushable.java:80)
>     at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$$Lambda$51/1152642867.run(Unknown
Source)
>     at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$runInParallel$3(SuperActivityOperatorNodePushable.java:202)
>     at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$$Lambda$52/69643057.call(Unknown
Source)
>     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>     ... 3 more 
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: java.nio.channels.ClosedChannelException
>     at org.apache.hyracks.control.nc.io.IOManager.syncRead(IOManager.java:209)
>     at org.apache.hyracks.storage.common.buffercache.BufferCache.read(BufferCache.java:537)
>     at org.apache.hyracks.storage.common.buffercache.BufferCache.pin(BufferCache.java:213)
>     at org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:562)
>     at org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:593)
>     at org.apache.hyracks.storage.am.btree.impls.BTree.search(BTree.java:199)
>     at org.apache.hyracks.storage.am.btree.impls.BTree.access$200(BTree.java:68)
>     at org.apache.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.search(BTree.java:883)
>     at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTreePointSearchCursor.hasNext(LSMBTreePointSearchCursor.java:71)
>     at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTreeSearchCursor.hasNext(LSMBTreeSearchCursor.java:59)
>     at org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.writeSearchResults(IndexSearchOperatorNodePushable.java:155)
>     at org.apache.hyracks.storage.am.common.dataflow.IndexSearchOperatorNodePushable.nextFrame(IndexSearchOperatorNodePushable.java:191)
>     ... 13 more
> Caused by: java.nio.channels.ClosedChannelException
>     at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:94)
>     at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:673)
>     at org.apache.hyracks.control.nc.io.IOManager.syncRead(IOManager.java:197)
>     ... 24 more
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message