hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lefty Leverenz (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-11301) thrift metastore issue when getting stats results in disconnect
Date Thu, 10 Sep 2015 07:45:45 GMT

    [ https://issues.apache.org/jira/browse/HIVE-11301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14738352#comment-14738352
] 

Lefty Leverenz commented on HIVE-11301:
---------------------------------------

[~pxiong], versions 1.2.0 and 1.2.1 are already released, so any new commits to branch-1.2
will go into a future version 1.2.2.  That's why I think Fix Version/s should say 1.2.2 instead
of 1.2.0.

> thrift metastore issue when getting stats results in disconnect
> ---------------------------------------------------------------
>
>                 Key: HIVE-11301
>                 URL: https://issues.apache.org/jira/browse/HIVE-11301
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>            Reporter: Sergey Shelukhin
>            Assignee: Pengcheng Xiong
>             Fix For: 1.2.0, 1.3.0, 2.0.0
>
>         Attachments: HIVE-11301.01.patch, HIVE-11301.02.patch
>
>
> On metastore side it looks like this:
> {noformat}
> 2015-07-17 20:32:27,795 ERROR [pool-3-thread-150]: server.TThreadPoolServer (TThreadPoolServer.java:run(294))
- Thrift error occurred during processing of message.
> org.apache.thrift.protocol.TProtocolException: Required field 'colStats' is unset! Struct:AggrStats(colStats:null,
partsFound:0)
>         at org.apache.hadoop.hive.metastore.api.AggrStats.validate(AggrStats.java:389)
>         at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_aggr_stats_for_result.validate(ThriftHiveMetastore.java)
>         at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_aggr_stats_for_result$get_aggr_stats_for_resultStandardScheme.write(ThriftHiveMetastore.java)
>         at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_aggr_stats_for_result$get_aggr_stats_for_resultStandardScheme.write(ThriftHiveMetastore.java)
>         at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_aggr_stats_for_result.write(ThriftHiveMetastore.java)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:53)
>         at org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:110)
>         at org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:106)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:422)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
>         at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:118)
>         at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> {noformat}
> and then
> {noformat}
> 2015-07-17 20:32:27,796 WARN  [pool-3-thread-150]: transport.TIOStreamTransport (TIOStreamTransport.java:close(112))
- Error closing output stream.
> java.net.SocketException: Socket closed
>         at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:116)
>         at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
>         at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>         at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
>         at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
>         at org.apache.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:110)
>         at org.apache.thrift.transport.TSocket.close(TSocket.java:196)
>         at org.apache.hadoop.hive.thrift.TFilterTransport.close(TFilterTransport.java:52)
>         at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:304)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> {noformat}
> Which on client manifests as
> {noformat}
> 2015-07-17 20:32:27,796 WARN  [main()]: metastore.RetryingMetaStoreClient (RetryingMetaStoreClient.java:invoke(187))
- MetaStoreClient lost connection. Attempting to reconnect.
> org.apache.thrift.transport.TTransportException
>         at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
>         at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
>         at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
>         at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
>         at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
>         at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
>         at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_aggr_stats_for(ThriftHiveMetastore.java:3029)
>         at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_aggr_stats_for(ThriftHiveMetastore.java:3016)
>         at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getAggrColStatsFor(HiveMetaStoreClient.java:2067)
>         at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:497)
>         at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:156)
>         at com.sun.proxy.$Proxy12.getAggrColStatsFor(Unknown Source)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:497)
>         at org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:1991)
>         at com.sun.proxy.$Proxy12.getAggrColStatsFor(Unknown Source)
>         at org.apache.hadoop.hive.ql.metadata.Hive.getAggrColStatsFor(Hive.java:3124)
>         at org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:252)
>         at org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:137)
>         at org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:125)
>         at org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$TableScanStatsRule.process(Stat
> sRulesProcFactory.java:111)
> {noformat}
> and CLI hangs for a really long time while this thing is retrying.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message