hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Bapat (Jira)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-22546) Postgres schema not using quoted identifiers for certain tables
Date Thu, 19 Dec 2019 06:03:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-22546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16999759#comment-16999759
] 

Ashutosh Bapat commented on HIVE-22546:
---------------------------------------

[~pvary], [~zchovan], this change is causing following stack trace when I run Hive with PostgreSQL
as backend db for the metastore.

0: jdbc:hive2://localhost:10000> create database dumpdb with ('repl.source.for'='1,2,3');0:
jdbc:hive2://localhost:10000> create database dumpdb with ('repl.source.for'='1,2,3');Error:
Error while compiling statement: FAILED: ParseException line 1:28 missing KW_DBPROPERTIES
at '(' near '<EOF>' (state=42000,code=40000)0: jdbc:hive2://localhost:10000> create
database dumpdb with dbproperties ('repl.source.for'='1,2,3');ERROR : FAILED: Hive Internal
Error: org.apache.hadoop.hive.ql.lockmgr.LockException(Error communicating with the metastore)org.apache.hadoop.hive.ql.lockmgr.LockException:
Error communicating with the metastore at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.commitTxn(DbTxnManager.java:541)
at org.apache.hadoop.hive.ql.Driver.releaseLocksAndCommitOrRollback(Driver.java:687) at org.apache.hadoop.hive.ql.Driver.releaseLocksAndCommitOrRollback(Driver.java:653)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:969)

... stack trace clipped

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)Caused
by: MetaException(message:Unable to update transaction database org.postgresql.util.PSQLException:
ERROR: relation "materialization_rebuild_locks" does not exist  Position: 13 at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183) at
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308) at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365) at 

This happens because the table names in all the queries in TxnHandler.java (including the
one at 1312, which causes this stack trace) are not quoting the table names. I think we need
to go through all these queries and quote the tablenames and column names there. Just this
change won't suffice.

I have opened HIVE-22663 for the same. I have left the assignee automatic since I am not
sure who should work on this.

> Postgres schema not using quoted identifiers for certain tables
> ---------------------------------------------------------------
>
>                 Key: HIVE-22546
>                 URL: https://issues.apache.org/jira/browse/HIVE-22546
>             Project: Hive
>          Issue Type: Bug
>          Components: Database/Schema
>    Affects Versions: 4.0.0
>            Reporter: Zoltan Chovan
>            Assignee: Zoltan Chovan
>            Priority: Major
>             Fix For: 4.0.0
>
>         Attachments: HIVE-22546.2.patch, HIVE-22546.3.patch, HIVE-22546.3.patch, HIVE-22546.4.patch,
HIVE-22546.5.patch, HIVE-22546.6.patch, HIVE-22546.patch
>
>
> In the latest postgresql schema (standalone-metastore/metastore-server/src/main/sql/postgres/hive-schema-4.0.0.postgres.sql)
the following tables have lowercase table and column names:
> {code:java}
> aux_table 
> compaction_queue 
> completed_compactions 
> completed_txn_components 
> hive_locks 
> materialization_rebuild_locks 
> min_history_level 
> next_compaction_queue_id 
> next_lock_id 
> next_txn_id 
> next_write_id 
> repl_txn_map 
> runtime_stats 
> txn_components 
> txn_to_write_id 
> txns 
> write_set{code}
> As these tables are referenced from the Hive sys database, the queries to these tables
will fail with a "Table not found" error.
> The problem is that the table and column names are not enclosed in quotes, so postgres
will turn these identifiers into lowercase.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message