hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vineet Mishra <clearmido...@gmail.com>
Subject Re: HiveServer2 Goes down while parallel Query execution
Date Sat, 10 Oct 2015 03:55:14 GMT
Hi Jimmy,

Any prominent reason for it run out of worker thread when maximum query
fired at a time fired are only 8.

I am trying to open multiple session on different connection, so each
thread corresponds to one connection.

Thanks!

On Sat, Oct 10, 2015 at 1:22 AM, Jimmy Xiang <jxiang@cloudera.com> wrote:

> Could it be because HS2 runs out of worker threads?
> Are you trying to open multiple sessions on the same connection?
>
> Thanks,
> Jimmy
>
> On Fri, Oct 9, 2015 at 12:34 PM, Vineet Mishra <clearmidoubt@gmail.com>
> wrote:
>
>> Any idea about this?
>>
>> Frequent Connectivity issue to HiveServer2
>>
>>   2015-10-10 00:23:11,070 [main] ERROR (HiveConnection.java:439) - Error
>> opening session
>>   org.apache.thrift.transport.TTransportException
>> at
>> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
>> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
>> at
>> org.apache.thrift.transport.TSaslTransport.readLength(TSaslTransport.java:355)
>> at
>> org.apache.thrift.transport.TSaslTransport.readFrame(TSaslTransport.java:432)
>> at
>> org.apache.thrift.transport.TSaslTransport.read(TSaslTransport.java:414)
>> at
>> org.apache.thrift.transport.TSaslClientTransport.read(TSaslClientTransport.java:37)
>> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
>> at
>> org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
>> at
>> org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
>> at
>> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
>> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
>> at
>> org.apache.hive.service.cli.thrift.TCLIService$Client.recv_OpenSession(TCLIService.java:160)
>> at
>> org.apache.hive.service.cli.thrift.TCLIService$Client.OpenSession(TCLIService.java:147)
>> at
>> org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:429)
>> at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:192)
>> at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
>> at java.sql.DriverManager.getConnection(DriverManager.java:571)
>> at java.sql.DriverManager.getConnection(DriverManager.java:215)
>> at com.sd.dwh.sc.tungsten.custom.HiveRunnable.<init>(HiveRunnable.java:42)
>> at com.sd.dwh.sc.tungsten.custom.HiveInvoker.main(HiveInvoker.java:62)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
>> 2015-10-10 00:23:11,071 [main] ERROR (HiveRunnable.java:48) - Could not
>> establish connection to jdbc:hive2://hadoop-hs2:10000/mydb: null
>>
>> URGENT CALL!
>>
>> Thanks!
>>
>> On Fri, Oct 9, 2015 at 2:42 PM, Vineet Mishra <clearmidoubt@gmail.com>
>> wrote:
>>
>>> This looks out to be the issue
>>>
>>> https://issues.apache.org/jira/browse/HIVE-2314
>>>
>>> Any workaround or resolution to the same.
>>>
>>> Thanks!
>>>
>>> On Fri, Oct 9, 2015 at 1:24 PM, Vineet Mishra <clearmidoubt@gmail.com>
>>> wrote:
>>>
>>>> Hi All,
>>>>
>>>> I am trying to connect to HiveServer2 to query some data in parallel
>>>> and landing up into some weird exception, stack trace mentioned below
>>>>
>>>> java.sql.SQLException: Error while cleaning up the server resources
>>>> at org.apache.hive.jdbc.HiveConnection.close(HiveConnection.java:569)
>>>> at
>>>> com.sd.dwh.sc.tungsten.custom.HiveRunnable.mergeJDBC(HiveRunnable.java:93)
>>>> at com.sd.dwh.sc.tungsten.custom.HiveRunnable.run(HiveRunnable.java:55)
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>> at java.lang.Thread.run(Thread.java:745)
>>>> Caused by: org.apache.thrift.transport.TTransportException
>>>> at
>>>> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
>>>> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
>>>> at
>>>> org.apache.thrift.transport.TSaslTransport.readLength(TSaslTransport.java:355)
>>>> at
>>>> org.apache.thrift.transport.TSaslTransport.readFrame(TSaslTransport.java:432)
>>>> at
>>>> org.apache.thrift.transport.TSaslTransport.read(TSaslTransport.java:414)
>>>> at
>>>> org.apache.thrift.transport.TSaslClientTransport.read(TSaslClientTransport.java:37)
>>>> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
>>>> at
>>>> org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
>>>> at
>>>> org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
>>>> at
>>>> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
>>>> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
>>>> at
>>>> org.apache.hive.service.cli.thrift.TCLIService$Client.recv_CloseSession(TCLIService.java:183)
>>>> at
>>>> org.apache.hive.service.cli.thrift.TCLIService$Client.CloseSession(TCLIService.java:170)
>>>> at org.apache.hive.jdbc.HiveConnection.close(HiveConnection.java:567)
>>>> ... 5 more
>>>>
>>>>
>>>> Any quick revert will be highly appreciable!
>>>>
>>>> Thanks!
>>>>
>>>>
>>>
>> --
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "CDH Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to cdh-user+unsubscribe@cloudera.org.
>> For more options, visit https://groups.google.com/a/cloudera.org/d/optout
>> .
>>
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "CDH Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cdh-user+unsubscribe@cloudera.org.
> For more options, visit https://groups.google.com/a/cloudera.org/d/optout.
>

Mime
View raw message