drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Khurram Faraaz (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-3743) query hangs on sqlline once Drillbit on foreman node is killed
Date Sat, 05 Sep 2015 02:10:45 GMT
Khurram Faraaz created DRILL-3743:
-------------------------------------

             Summary: query hangs on sqlline once Drillbit on foreman node is killed
                 Key: DRILL-3743
                 URL: https://issues.apache.org/jira/browse/DRILL-3743
             Project: Apache Drill
          Issue Type: Bug
          Components: Execution - Flow
    Affects Versions: 1.2.0
         Environment: 4 node cluster CentOS
            Reporter: Khurram Faraaz
            Assignee: Chris Westin


sqlline/query hangs once Drillbit (on Foreman node) is killed. (kill -9 <PID>)
query was issued from the Foreman node. The query returns many records, and it is a long running
query.

Steps to reproduce the problem.

set planner.slice_target=1

1.  clush -g khurram service mapr-warden stop
2.  clush -g khurram service mapr-warden start
3.  ./sqlline -u "jdbc:drill:schema=dfs.tmp"
0: jdbc:drill:schema=dfs.tmp> select * from `twoKeyJsn.json` limit 2000000;

4.  Immediately from another console do a jps and kill the Drillbit process (in this case
foreman) while the query is being run on sqlline. You will notice that sqlline just hangs,
we do not see any exceptions or errors being reported on sqlline prompt or in drillbit.log
or drillbit.out

I do see this Exception in sqlline.log on the node from where sqlline was started

{code}
2015-09-04 18:45:12,069 [Client-1] INFO  o.a.d.e.rpc.user.QueryResultHandler - User Error
Occurred
org.apache.drill.common.exceptions.UserException: CONNECTION ERROR: Connection /10.10.100.201:53425
<--> /10.10.100.201:31010 (user client) closed unexpectedly.


[Error Id: ec316cfd-c9a5-4905-98e3-da20cb799ba5 ]
        at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:524)
~[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
        at org.apache.drill.exec.rpc.user.QueryResultHandler$SubmissionListener$ChannelClosedListener.operationComplete(QueryResultHandler.java:298)
[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
        at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
        at io.netty.util.concurrent.DefaultPromise$LateListeners.run(DefaultPromise.java:845)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
        at io.netty.util.concurrent.DefaultPromise$LateListenerNotifier.run(DefaultPromise.java:873)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:254) [netty-transport-native-epoll-4.0.27.Final-linux-x86_64.jar:na]
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
        at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
2015-09-04 18:45:12,069 [Client-1] INFO  o.a.d.j.i.DrillResultSetImpl$ResultsListener - [#7]
Query failed:
org.apache.drill.common.exceptions.UserException: CONNECTION ERROR: Connection /10.10.100.201:53425
<--> /10.10.100.201:31010 (user client) closed unexpectedly.


[Error Id: ec316cfd-c9a5-4905-98e3-da20cb799ba5 ]
        at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:524)
~[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
        at org.apache.drill.exec.rpc.user.QueryResultHandler$SubmissionListener$ChannelClosedListener.operationComplete(QueryResultHandler.java:298)
[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
        at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
        at io.netty.util.concurrent.DefaultPromise$LateListeners.run(DefaultPromise.java:845)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
        at io.netty.util.concurrent.DefaultPromise$LateListenerNotifier.run(DefaultPromise.java:873)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:254) [netty-transport-native-epoll-4.0.27.Final-linux-x86_64.jar:na]
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
        at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
2015-09-04 18:45:12,071 [Client-1] ERROR o.a.d.e.rpc.user.QueryResultHandler - SYSTEM ERROR:
ChannelClosedException


[Error Id: c53c477f-f1cf-4458-8620-b1e11ba31701 ]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: ChannelClosedException


[Error Id: c53c477f-f1cf-4458-8620-b1e11ba31701 ]
        at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:524)
~[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
        at org.apache.drill.exec.rpc.user.QueryResultHandler$SubmissionListener.failed(QueryResultHandler.java:312)
[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
        at org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.setException(CoordinationQueue.java:103)
[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
        at org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:89)
[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
        at org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:67)
[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
        at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
        at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
        at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
        at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:424) [netty-common-4.0.27.Final.jar:4.0.27.Final]
        at io.netty.channel.AbstractChannel$AbstractUnsafe.safeSetFailure(AbstractChannel.java:788)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
        at io.netty.channel.AbstractChannel$AbstractUnsafe.write(AbstractChannel.java:689)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.write(DefaultChannelPipeline.java:1114)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:705)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
        at io.netty.channel.AbstractChannelHandlerContext.access$1900(AbstractChannelHandlerContext.java:32)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
        at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.write(AbstractChannelHandlerContext.java:980)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
        at io.netty.channel.AbstractChannelHandlerContext$WriteAndFlushTask.write(AbstractChannelHandlerContext.java:1032)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
        at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.run(AbstractChannelHandlerContext.java:965)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:254) [netty-transport-native-epoll-4.0.27.Final-linux-x86_64.jar:na]
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
        at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
Caused by: org.apache.drill.exec.rpc.ChannelClosedException: null
        ... 18 common frames omitted
{code}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message