hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-5264) SQL generated by MetaStoreDirectSql.java not compliant with Postgres.
Date Wed, 11 Sep 2013 00:09:52 GMT

    [ https://issues.apache.org/jira/browse/HIVE-5264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13763746#comment-13763746
] 

Sergey Shelukhin commented on HIVE-5264:
----------------------------------------

In this case, metastore is supposed to fall back to JDO. Doesn't the fallback work? I seem
to have verified it in the past.

I am not really certain what can be done about the casing issue. We could try to use lower
case names in the db itself, but it would break compat, all tables would need to be renamed.
I would really like to avoid reimplementing MetaStoreDirectSql with code for specific databases,
"faster datanucleus with much less features". After all it's just a perf optimization...
Does Postgres have some sort of ANSI-compliance flag? It might help.
                
> SQL generated by MetaStoreDirectSql.java not compliant with Postgres.
> ---------------------------------------------------------------------
>
>                 Key: HIVE-5264
>                 URL: https://issues.apache.org/jira/browse/HIVE-5264
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>    Affects Versions: 0.11.0
>         Environment: Ubuntu 12.04
> PostgreSQL 9.1.8
>            Reporter: Alexander Behm
>
> Some operations against the Hive Metastore seem broken
> against Postgres.
> For example, when using HiveMetastoreClient.listPartitions()
> the Postgres logs show queries such as:
> 2013-09-09 19:10:01 PDT STATEMENT:  select PARTITIONS.PART_ID from
> PARTITIONS  inner join TBLS on PARTITIONS.TBL_ID = TBLS.TBL_ID   inner
> join DBS on TBLS.DB_ID = DBS.DB_ID  where TBLS.TBL_NAME = $1 and
> DBS.NAME = $2 order by PART_NAME asc
> with a somewhat cryptic (but correct) error:
> ERROR:  relation "partitions" does not exist at character 32
> Postgres identifiers are somewhat unusual. Unquoted identifiers are interpreted as lower
case (there is no Postgres option to change this). Since the Metastore table schema uses upper
case table names, the correct SQL requires escaped identifiers to those tables, i.e.,
> select "PARTITIONS"."PART_ID" from "PARTITIONS"...
> Hive sets metastore.try.direct.sql=true by default, so the above SQL is generated by
hive/metastore/MetaStoreDirectSql.java, i.e., this is not a Datanucleus problem.
> When I set metastore.try.direct.sql=false, then the Metastore backed by Postgres works.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message