drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sudheesh Katkam <skat...@maprtech.com>
Subject Re: Channel closed exception
Date Thu, 01 Oct 2015 04:31:09 GMT
Ah there, I assumed you were not on Windows. We do not have the drillbit.out file for Windows
(I think).

Anyway, the error messages you see are symptoms of the real issue, which is most likely an
“out of memory” scenario. I think that the drillbit did not crash either (in which case
you might see a JVM core dump). Instead the drillbit gracefully shutdown unable to handle
the error. Could you share the query (if possible), data set details, and configuration?

It would also help if you put the entire log files somewhere (Github gist or Dropbox) and
shared the links.

- Sudheesh

> On Sep 30, 2015, at 9:10 PM, Sudip Mukherjee <smukherjee@commvault.com> wrote:
> 
> Yes. I was running drill in embedded mode to try it out with  3.9 lakhs records , near
about 50+ columns.[ sqlline.bat -u "jdbc:drill:zk=local" ]
> I see sqlline.log and sqlline_queries.log under log folder. Did I miss some steps?
> 
> But was able run queries from the command prompt after that
> 
> Thanks,
> Sudip
> 
> -----Original Message-----
> From: Sudheesh Katkam [mailto:skatkam@maprtech.com] 
> Sent: 01 October 2015 AM 09:24
> To: dev@drill.apache.org
> Subject: Re: Channel closed exception
> 
> Are you running drill in embedded mode? The directory containing the drillbit.log file
contains the drillbit.out file too. There might be a JVM core dump in the /tmp directory.
> 
> Usually "Received shutdown request” message means that’s the drillbit ran out of
memory. Given that you mentioned the query is on a large document set, this scenario is likely.
> 
> - Sudheesh
> 
>> On Sep 30, 2015, at 8:46 PM, Sudip Mukherjee <smukherjee@commvault.com> wrote:
>> 
>> Hi,
>> This is the excerpt of exception in sqlline.log. Could you please refer me to the
location of drillbit.out file?
>> 
>> 
>> 2015-09-30 22:23:36,286 [BitServer-4] WARN  o.a.d.exec.rpc.control.ControlServer
- Message of mode REQUEST of rpc type 6 took longer than 500ms.  Actual duration was 16047ms.
>> 2015-09-30 22:24:13,581 
>> [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO  
>> o.a.d.e.w.fragment.FragmentExecutor - 
>> 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State change requested 
>> CANCELLATION_REQUESTED --> FINISHED
>> 2015-09-30 22:24:13,581 
>> [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO  
>> o.a.d.e.w.f.FragmentStatusReporter - 
>> 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State to report: CANCELLED
>> 2015-09-30 22:24:13,581 [BitServer-4] INFO  
>> o.a.drill.exec.work.foreman.Foreman - State change requested.  
>> CANCELLATION_REQUESTED --> COMPLETED
>> 2015-09-30 22:24:13,581 [BitServer-4] INFO  o.a.drill.exec.work.foreman.Foreman -
foreman cleaning up.
>> 2015-09-30 22:24:13,597 [UserServer-1] INFO  o.a.drill.exec.work.foreman.Foreman
- Failure while trying communicate query result to initiating client. This would happen if
a client is disconnected before response notice can be sent.
>> org.apache.drill.exec.rpc.ChannelClosedException: null
>> 	at org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:89)
[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
>> 	at org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:67)
[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
>> 	at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
>> 	at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
>> 	at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
>> 	at 
>> io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java
>> :424) [netty-common-4.0.27.Final.jar:4.0.27.Final]
>> 
>> Thanks,
>> Sudip
>> 
>> -----Original Message-----
>> From: Sudheesh Katkam [mailto:skatkam@maprtech.com]
>> Sent: 01 October 2015 AM 09:09
>> To: dev@drill.apache.org
>> Subject: Re: Channel closed exception
>> 
>> Can you check if there are any error messages in the drillbit.out file?
>> 
>> - Sudheesh
>> 
>>> On Sep 30, 2015, at 8:35 PM, Sudip Mukherjee <smukherjee@commvault.com>
wrote:
>>> 
>>> Hi Devs,
>>> 
>>> Getting into a channel closed exception when I tried to connect drill to tableau
and query a large document set. Could it be because tableau dropped the connection due to
datasize. Any thoughts?
>>> 
>>> 2015-09-30 21:13:43,716
>>> [29f3777a-d407-295a-d45e-243fb70efc07:frag:0:0] INFO 
>>> o.a.d.e.w.fragment.FragmentExecutor -
>>> 29f3777a-d407-295a-d45e-243fb70efc07:0:0: State change requested 
>>> AWAITING_ALLOCATION --> RUNNING
>>> 2015-09-30 21:13:43,716
>>> [29f3777a-d407-295a-d45e-243fb70efc07:frag:0:0] INFO 
>>> o.a.d.e.w.f.FragmentStatusReporter -
>>> 29f3777a-d407-295a-d45e-243fb70efc07:0:0: State to report: RUNNING
>>> 2015-09-30 21:14:36,875
>>> [29f3777a-d407-295a-d45e-243fb70efc07:frag:0:0] INFO 
>>> o.a.d.e.w.fragment.FragmentExecutor -
>>> 29f3777a-d407-295a-d45e-243fb70efc07:0:0: State change requested 
>>> RUNNING --> FINISHED
>>> 2015-09-30 21:14:36,876
>>> [29f3777a-d407-295a-d45e-243fb70efc07:frag:0:0] INFO 
>>> o.a.d.e.w.f.FragmentStatusReporter -
>>> 29f3777a-d407-295a-d45e-243fb70efc07:0:0: State to report: FINISHED
>>> 2015-09-30 21:14:36,877 [BitServer-4] INFO 
>>> o.a.drill.exec.work.foreman.Foreman - State change requested.  
>>> RUNNING
>>> --> COMPLETED
>>> 2015-09-30 21:14:36,877 [BitServer-4] INFO  o.a.drill.exec.work.foreman.Foreman
- foreman cleaning up.
>>> 2015-09-30 21:33:09,745 [Client-1] INFO  o.a.drill.exec.rpc.user.UserClient -
Channel closed /172.19.77.101:57162 <--> DEVSTAGING2.testlab.commvault.com/172.19.77.101:31010.
>>> 2015-09-30 21:33:12,859 [BitServer-3] INFO  o.a.d.exec.rpc.control.ControlClient
- Channel closed /172.19.77.101:57207 <--> DEVSTAGING2.testlab.commvault.com/172.19.77.101:31011.
>>> 2015-09-30 21:33:16,077 [main] INFO  o.apache.drill.exec.server.Drillbit - Shutdown
completed (6331 ms).
>>> 2015-09-30 21:33:16,102 [Drillbit-ShutdownHook#0] INFO  o.apache.drill.exec.server.Drillbit
- Received shutdown request.
>>> 
>>> Thanks,
>>> Sudip
>>> 
>>> 
>>> 
>>> ***************************Legal 
>>> Disclaimer***************************
>>> "This communication may contain confidential and privileged material 
>>> for the sole use of the intended recipient. Any unauthorized review, 
>>> use or distribution by others is strictly prohibited. If you have 
>>> received the message by mistake, please advise the sender by reply email and
delete the message. Thank you."
>>> *********************************************************************
>>> *
>> 
>> 
>> 
>> 
>> 
>> ***************************Legal Disclaimer***************************
>> "This communication may contain confidential and privileged material 
>> for the sole use of the intended recipient. Any unauthorized review, 
>> use or distribution by others is strictly prohibited. If you have 
>> received the message by mistake, please advise the sender by reply email and delete
the message. Thank you."
>> **********************************************************************
> 
> 
> 
> 
> 
> ***************************Legal Disclaimer***************************
> "This communication may contain confidential and privileged material for the
> sole use of the intended recipient. Any unauthorized review, use or distribution
> by others is strictly prohibited. If you have received the message by mistake,
> please advise the sender by reply email and delete the message. Thank you."
> **********************************************************************


Mime
View raw message