hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Csaba Németi (JIRA) <j...@apache.org>
Subject [jira] [Commented] (HIVE-7594) Hive JDBC client: "out of sequence response" on large long running query
Date Fri, 07 Aug 2015 13:57:45 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14661855#comment-14661855
] 

Csaba Németi commented on HIVE-7594:
------------------------------------

Hi. I got the same error running a simple query that doesn't take minutes...
What I discovered (I'm 1 month old in Hadoop & Hive so, take this in account please) is
the following: I think SQL processing is asynchronous and this raise serious problems if you
try to run two queries in parallel. So the way I fixed the problem was to synchronize the
methods using Hive connection.

> Hive JDBC client: "out of sequence response" on large long running query
> ------------------------------------------------------------------------
>
>                 Key: HIVE-7594
>                 URL: https://issues.apache.org/jira/browse/HIVE-7594
>             Project: Hive
>          Issue Type: Bug
>          Components: Clients, HiveServer2
>    Affects Versions: 0.13.0
>         Environment: HDP2.1
>            Reporter: Hari Sekhon
>
> When executing a long running query in a JDBC client (Squirrel) to HiveServer2 after
several minutes I get this error in the client:
> {code}
> Error: org.apache.thrift.TApplicationException: ExecuteStatement failed: out of sequence
response
> SQLState:  08S01
> ErrorCode: 0
> {code}
> I've seen this before in, iirc when running 2 queries in 1 session but I've closed the
client and run only this single query in a new session each time. I did a search and saw HIVE-6893
referring to a Metastore exception which I have in some older logs but not corresponding /
recent in these recent instances, the error seems different in this case but may be related.
> The query to reproduce is "select count(*) from myTable" where myTable is a 1TB table
of 620 million rows. This happens in both MR and Tez execution engines running on Yarn.
> Here are all the jars I've added to the classpath (taken from Hortonworks doc http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.1-latest/bk_dataintegration/content/ch_using-hive-2.html,
plus added hadoop-common, hive-exec and slf4j-api to solve class not found issues on top of
that):
> commons-codec-1.4.jar
> commons-logging-1.1.3.jar
> hadoop-common-2.4.0.2.1.3.0-563.jar
> hive-exec-0.13.0.2.1.3.0-563.jar
> hive-jdbc-0.13.0.2.1.3.0-563.jar
> hive-service-0.13.0.2.1.3.0-563.jar
> httpclient-4.2.5.jar
> httpcore-4.2.5.jar
> libthrift-0.9.0.jar
> slf4j-api-1.7.5.jar
> I am seeing errors like this in the hiveserver2.log:
> {code}
> 2014-08-01 15:04:31,358 ERROR [pool-5-thread-3]: server.TThreadPoolServer (TThreadPoolServer.java:run(215))
- Error occurred during processing of message.
> java.lang.RuntimeException: org.apache.thrift.transport.TTransportException
>         at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
>         at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:189)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.thrift.transport.TTransportException
>         at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
>         at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
>         at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:182)
>         at org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125)
>         at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:253)
>         at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
>         at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
>         ... 4 more
> ...
> 2014-08-01 15:06:31,520 ERROR [pool-5-thread-3]: server.TThreadPoolServer (TThreadPoolServer.java:run(215))
- Error occurred during processing of message.
> java.lang.RuntimeException: org.apache.thrift.transport.TTransportException
>         at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
>         at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:189)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.thrift.transport.TTransportException
>         at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
>         at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
>         at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:182)
>         at org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125)
>         at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:253)
>         at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
>         at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
>         ... 4 more
> ...
> 2014-08-01 15:05:41,096 ERROR [pool-4-thread-22]: operation.Operation (SQLOperation.java:run(202))
- Error running hive query:
> org.apache.hive.service.cli.HiveSQLException: Illegal Operation state transition
>         at org.apache.hive.service.cli.OperationState.validateTransition(OperationState.java:94)
>         at org.apache.hive.service.cli.OperationState.validateTransition(OperationState.java:99)
>         at org.apache.hive.service.cli.operation.Operation.setState(Operation.java:109)
>         at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:158)
>         at org.apache.hive.service.cli.operation.SQLOperation.access$000(SQLOperation.java:68)
>         at org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:199)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
>         at org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:500)
>         at org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:208)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> {code}
> Regards,
> Hari Sekhon
> http://www.linkedin.com/in/harisekhon



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message