hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Koifman (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-7811) Compactions need to update table/partition stats
Date Tue, 02 Sep 2014 21:20:23 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14118769#comment-14118769
] 

Eugene Koifman commented on HIVE-7811:
--------------------------------------

Actually, it only updates stats on major compactions, but the code is confusing.  I added
patch 6 with minor refactoring to make this cleaner.  

Unfortunately TestCompactor currently fails because stats data returned is duplicated.  It
seems that analyze command on update of stats does another insert instead of update so the
DB contains both the old stats and new stats.
Likely this was broken by HIVE-7876.

> Compactions need to update table/partition stats
> ------------------------------------------------
>
>                 Key: HIVE-7811
>                 URL: https://issues.apache.org/jira/browse/HIVE-7811
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Transactions
>    Affects Versions: 0.13.1
>            Reporter: Eugene Koifman
>            Assignee: Eugene Koifman
>         Attachments: HIVE-7811.3.patch, HIVE-7811.4.patch, HIVE-7811.5.patch, HIVE-7811.6.patch
>
>
> Compactions should trigger stats recalculation for columns which already have sats.
> https://reviews.apache.org/r/25201/
> Major compactions will cause the Compactor to see which columns already have stats and
run analyze command for those columns.  If compacting a partition then stats for that partition
will be computed.  If table is not partitioned, then the whole table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message