hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-5304) Hive results can depend on metastore's underlying datastore, if autocreate is used
Date Fri, 11 Oct 2013 22:20:42 GMT

     [ https://issues.apache.org/jira/browse/HIVE-5304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sergey Shelukhin updated HIVE-5304:
-----------------------------------

    Summary: Hive results can depend on metastore's underlying datastore, if autocreate is
used  (was: there are various string encoding problems in metastore depending on underlying
datastore)

> Hive results can depend on metastore's underlying datastore, if autocreate is used
> ----------------------------------------------------------------------------------
>
>                 Key: HIVE-5304
>                 URL: https://issues.apache.org/jira/browse/HIVE-5304
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>            Reporter: Sergey Shelukhin
>
> Hive uses JDOQL filters to optimize partition retrieval; recently direct SQL was added
to optimize it further. Both of these methods may end up pushing StringCol op 'SomeString'
to underlying SQL datastore. Many paths also pushes order by-s, although these are not as
problematic.
> The problem is that different datastores handle string compares differently. 
> While testing on Postgres, I see that results in different things, from innocent like
order changes in "show partitions", to more serious like 
> {code}
>  alter table ptestfilter drop partition (c>='US', d<='2')
> {code}
> in drop_partitions_filter.q - in Derby, with which the .q.out file was generated, it
drops "c=Uganda/d=2"; this also passes on MySQL (I ran tests with autocreated db); on Postgres
with a db from the script it doesn't.
> Looks like we need to enforce collation in partition names and part_key_values-es; both
in the create scripts, as well as during autocreate (via package.jdo?)
> EDIT:
> also affected are - show indexes. So all names need to be taken care of
> Then; describe_comment_nonascii.q fails against MySQL on autocreated db due to problems
with commends.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message