hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dudu Markovitz (JIRA)" <>
Subject [jira] [Commented] (HIVE-5483) use metastore statistics to optimize max/min/etc. queries
Date Sat, 30 Jul 2016 20:30:20 GMT


Dudu Markovitz commented on HIVE-5483:

Hi guys, perhaps I'm missing something but do I have any guarantee for the correctness of
the metadata when someone can simply delete or replace a files directly in the file system
without going through the metastore? 

> use metastore statistics to optimize max/min/etc. queries
> ---------------------------------------------------------
>                 Key: HIVE-5483
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Sergey Shelukhin
>            Assignee: Ashutosh Chauhan
>             Fix For: 0.13.0
>         Attachments: HIVE-5483.2.patch, HIVE-5483.3.patch, HIVE-5483.patch
> We have discussed this a little bit.
> Hive can answer queries such as select max(c1) from t purely from metastore using partition
statistics, provided that we know the statistics are up to date.
> All data changes (e.g. adding new partitions) currently go thru metastore so we can track
up-to-date-ness. If they are not up-to-date, the queries will have to read data (at least
for outdated partitions) until someone runs analyze table. We can also analyze new partitions
after add, if that is configured/specified in the command.

This message was sent by Atlassian JIRA

View raw message