asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Till Westmann (JIRA)" <j...@apache.org>
Subject [jira] [Reopened] (ASTERIXDB-1144) FeedMetaStoreNodePushable.close() call hangs
Date Thu, 19 Nov 2015 04:52:11 GMT

     [ https://issues.apache.org/jira/browse/ASTERIXDB-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Till Westmann reopened ASTERIXDB-1144:
--------------------------------------

Reopened while it's not in master.

> FeedMetaStoreNodePushable.close() call hangs
> --------------------------------------------
>
>                 Key: ASTERIXDB-1144
>                 URL: https://issues.apache.org/jira/browse/ASTERIXDB-1144
>             Project: Apache AsterixDB
>          Issue Type: Bug
>            Reporter: Young-Seok Kim
>            Assignee: Abdullah Alamoudi
>            Priority: Critical
>
> Feed job hangs in FeedMetaStoreNodePushable.close() call as shown in the following jstack
trace:
> "org.apache.hyracks.api.rewriter.runtime.SuperActivity:TAID:TID:ANID:ODID:1:0:7:0:0"
daemon prio=10 tid=0x00007fac6005c000 nid=0x4310 in Object.wait() [0x00007facd74f3000]
>    java.lang.Thread.State: WAITING (on object monitor)
>         at java.lang.Object.wait(Native Method)
>         at java.lang.Object.wait(Object.java:503)
>         at org.apache.asterix.metadata.feeds.FeedMetaStoreNodePushable.close(FeedMetaStoreNodePushable.java:195)
>         - locked <0x0000000677fd9e80> (a org.apache.asterix.common.dataflow.AsterixLSMInsertDeleteOperatorNodePushable)
>         at org.apache.hyracks.algebricks.runtime.operators.std.StreamProjectRuntimeFactory$1.close(StreamProjectRuntimeFactory.java:140)
>         at org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.close(AssignRuntimeFactory.java:220)
>         at org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.close(AlgebricksMetaOperatorDescriptor.java:145)
>         at org.apache.asterix.metadata.feeds.FeedMetaNodePushable.close(FeedMetaNodePushable.java:174)
>         at org.apache.hyracks.storage.am.common.dataflow.IndexInsertUpdateDeleteOperatorNodePushable.close(IndexInsertUpdateDeleteOperatorNodePushable.java:153)
>         at org.apache.asterix.metadata.feeds.FeedMetaStoreNodePushable.close(FeedMetaStoreNodePushable.java:200)
>         at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:349)
>         at org.apache.hyracks.control.nc.Task.run(Task.java:290)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> The reason of the hang seems to have a bug in the wait/notify code. 
> More specifically,  FrameEventCallback.frameEvent() method gives notification in the
following code snippet:
> ------------------------------------------------------
>             case FINISHED_PROCESSING:
>                 inputSideHandler.setFinished(true);
>                 synchronized (coreOperator) {
>                     coreOperator.notifyAll();
>                 }
> ------------------------------------------------------
> FeedMetaStoreNodePushable.close() methods waits notification in the following code snippet:
> ------------------------------------------------------
>                 while (!inputSideHandler.isFinished()) {
>                     synchronized (coreOperator) {
>                         coreOperator.wait();
>                     }
>                 }
> ------------------------------------------------------
> If a caller thread of the close() just called isFinished(), it's return value is false,
then the thread is scheduled by OS and waits for the next scheduling for running.
> Then, if the a caller thread of the frameEvent() called setFinished(true) and coreOperator.notifyAll(),
then the notification of notifyAll() can be lost. In other words, the notification may not
reach to the caller thread of the close(). 
> If this happens, the caller may hang as shown in the above jstack trace. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message