hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dudu Markovitz (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-5483) use metastore statistics to optimize max/min/etc. queries
Date Sat, 30 Jul 2016 20:30:20 GMT

    [ https://issues.apache.org/jira/browse/HIVE-5483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15400813#comment-15400813
] 

Dudu Markovitz commented on HIVE-5483:
--------------------------------------

Hi guys, perhaps I'm missing something but do I have any guarantee for the correctness of
the metadata when someone can simply delete or replace a files directly in the file system
without going through the metastore? 

> use metastore statistics to optimize max/min/etc. queries
> ---------------------------------------------------------
>
>                 Key: HIVE-5483
>                 URL: https://issues.apache.org/jira/browse/HIVE-5483
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Sergey Shelukhin
>            Assignee: Ashutosh Chauhan
>             Fix For: 0.13.0
>
>         Attachments: HIVE-5483.2.patch, HIVE-5483.3.patch, HIVE-5483.patch
>
>
> We have discussed this a little bit.
> Hive can answer queries such as select max(c1) from t purely from metastore using partition
statistics, provided that we know the statistics are up to date.
> All data changes (e.g. adding new partitions) currently go thru metastore so we can track
up-to-date-ness. If they are not up-to-date, the queries will have to read data (at least
for outdated partitions) until someone runs analyze table. We can also analyze new partitions
after add, if that is configured/specified in the command.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message