hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Tryuber (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-8627) Compute stats on a table from impala caused the table to be corrupted
Date Sat, 30 May 2015 15:45:17 GMT

    [ https://issues.apache.org/jira/browse/HIVE-8627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14566062#comment-14566062
] 

Sergey Tryuber commented on HIVE-8627:
--------------------------------------

For those who encounter similar issue (the patch has been backported by Cloudera to CDH5.2.1,
but the bug is still present in HDP2.2), the only way which I've found to recover Impala without
dropping tables is:
# Connect to Hive metastore database (in my case it was "mysql -h localhost -Dhive")
# Truncate column statistics ("_truncate table TAB_COL_STATS;_" command)
# Do not execute "COMPUTE STATS" until you're sure that your Hive has this patch applied.
Use Hive's native equivalents instead.

> Compute stats on a table from impala caused the table to be corrupted
> ---------------------------------------------------------------------
>
>                 Key: HIVE-8627
>                 URL: https://issues.apache.org/jira/browse/HIVE-8627
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>    Affects Versions: 0.13.0, 0.13.1
>            Reporter: Na Yang
>            Assignee: Na Yang
>             Fix For: 1.1.0
>
>         Attachments: HIVE-8627.2.patch, HIVE-8627.patch
>
>
> Use impala 2.0 to connect to hive-0.13 Metastore.
> From impala, run the following queries:
> {noformat}
> create table voter1(voter_id int,name string,age tinyint, registration
> string,contributions decimal(5,2),voterzone smallint,create_time timestamp) row
> format delimited fields terminated by '\t';
> load data inpath '/tmp/votertab' into table voter1;
> {noformat}
> After this, can successfully select from table voter 1.
> Execute the following from impala shell:
> {noformat}
> > compute stats voter1;
> {noformat}
> After this, got the following error selecting from table voter1:
> {noformat}
> > select * from voter1 limit 5;
> Query: select * from voter1 limit 5
> ERROR: AnalysisException: Failed to load metadata for table: default.voter1
> CAUSED BY: TableLoadingException: Failed to load metadata for table: voter1
> CAUSED BY: TTransportException: java.net.SocketException: Broken pipe
> CAUSED BY: SocketException: Broken pipe
> {noformat}
> Below is the exception found in Hive log:
> {noformat}
> org.apache.thrift.protocol.TProtocolException: Cannot write a TUnion with no set value!
> at org.apache.thrift.TUnion$TUnionStandardScheme.write(TUnion.java:240)
> at org.apache.thrift.TUnion$TUnionStandardScheme.write(TUnion.java:213)
> at org.apache.thrift.TUnion.write(TUnion.java:152)
> at org.apache.hadoop.hive.metastore.api.ColumnStatisticsObj$ColumnStatisticsObjStandardScheme.write(ColumnStatisticsObj.java:550)
> at org.apache.hadoop.hive.metastore.api.ColumnStatisticsObj$ColumnStatisticsObjStandardScheme.write(ColumnStatisticsObj.java:488)
> at org.apache.hadoop.hive.metastore.api.ColumnStatisticsObj.write(ColumnStatisticsObj.java:414)
> at org.apache.hadoop.hive.metastore.api.TableStatsResult$TableStatsResultStandardScheme.write(TableStatsResult.java:388)
> at org.apache.hadoop.hive.metastore.api.TableStatsResult$TableStatsResultStandardScheme.write(TableStatsResult.java:338)
> at org.apache.hadoop.hive.metastore.api.TableStatsResult.write(TableStatsResult.java:288)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_statistics_req_result$get_table_statistics_req_resultStandardScheme.write(ThriftHiveMetastore.java)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_statistics_req_result$get_table_statistics_req_resultStandardScheme.write(ThriftHiveMetastore.java)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_statistics_req_result.write(ThriftHiveMetastore.java)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:53)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at org.apache.hadoop.hive.metastore.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:48)
> at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message