hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Julien Lehuen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-6308) COLUMNS_V2 Metastore table not populated for tables created without an explicit column list.
Date Wed, 23 Jul 2014 06:40:39 GMT

    [ https://issues.apache.org/jira/browse/HIVE-6308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14071422#comment-14071422
] 

Julien Lehuen commented on HIVE-6308:
-------------------------------------

This is *critical* for Impala users! We want to be able to perform our Metastore operations
via Hive's Thrift server, but this would create unhealthy tables for which we can not run
ANALYZE. Such table's statistics cannot computed by Impala (statistics are absolutely necessary
for Impala to run joins in an optimal manner), and this means that we need to create table
via Impala over JDBC. This is less than ideal, and this issue is the only blocker from letting
us create via Hive over Thrift.

Related IMPALA tickets: [IMPALA-867|https://issues.cloudera.org/browse/IMPALA-867], [IMPALA-1104|https://issues.cloudera.org/browse/IMPALA-1104]

> COLUMNS_V2 Metastore table not populated for tables created without an explicit column
list.
> --------------------------------------------------------------------------------------------
>
>                 Key: HIVE-6308
>                 URL: https://issues.apache.org/jira/browse/HIVE-6308
>             Project: Hive
>          Issue Type: Bug
>          Components: Database/Schema
>    Affects Versions: 0.10.0
>            Reporter: Alexander Behm
>
> Consider this example table:
> CREATE TABLE avro_test
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
> STORED as INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
> TBLPROPERTIES (
> 'avro.schema.url'='file:///path/to/the/schema/test_serializer.avsc');
> When I try to run an ANALYZE TABLE for computing column stats on any of the columns,
then I get:
> org.apache.hadoop.hive.ql.metadata.HiveException: NoSuchObjectException(message:Column
o_orderpriority for which stats gathering is requested doesn't exist.)
>     at org.apache.hadoop.hive.ql.metadata.Hive.updateTableColumnStatistics(Hive.java:2280)
>     at org.apache.hadoop.hive.ql.exec.ColumnStatsTask.persistTableStats(ColumnStatsTask.java:331)
>     at org.apache.hadoop.hive.ql.exec.ColumnStatsTask.execute(ColumnStatsTask.java:343)
>     at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:138)
>     at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:66)
>     at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1383)
>     at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1169)
>     at org.apache.hadoop.hive.ql.Driver.run(Driver.java:982)
>     at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902)
>     at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>     at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>     at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
>     at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
>     at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:606)
>     at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> The root cause appears to be that the COLUMNS_V2 table in the Metastore isn't populated
properly during the table creation.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message