hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-20219) analyze and analyze for columns don't manage stats state correctly
Date Fri, 20 Jul 2018 22:22:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-20219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sergey Shelukhin updated HIVE-20219:
------------------------------------
    Description: 
Discovered while adding tests in HIVE-20029

Step 1. Have a table with fully correct txn stats.
Step 2. Run some query that sets stats to invalid (e.g. parallel inserts).
Step 3. Run analyze without for columns...

Result is that while only basic stats are updated, the flag is set to true with old column
stats values.
I blame json; if stats state was stored in proper places this kind of bug would be very unlikely.
Not sure if this would affect master, I'll add a test; ACID stats invalidation definitely
needs to invalidate all the stats, not just basic stats flag, when invalidating.

  was:
Discovered while adding tests in HIVE-20029

Step 1. Have a table with fully correct stats.
Step 2. Run some query that sets stats to invalid.
Step 3. Run analyze without for columns...

Result is that while only basic stats are updated, the flag is set to true with old column
stats values.
I blame json.
Not sure if this would affect master, I'll add a test; ACID stats invalidation definitely
needs to invalidate all the stats, not just basic stats flag, when invalidating.


> analyze and analyze for columns don't manage stats state correctly
> ------------------------------------------------------------------
>
>                 Key: HIVE-20219
>                 URL: https://issues.apache.org/jira/browse/HIVE-20219
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>            Priority: Major
>
> Discovered while adding tests in HIVE-20029
> Step 1. Have a table with fully correct txn stats.
> Step 2. Run some query that sets stats to invalid (e.g. parallel inserts).
> Step 3. Run analyze without for columns...
> Result is that while only basic stats are updated, the flag is set to true with old column
stats values.
> I blame json; if stats state was stored in proper places this kind of bug would be very
unlikely.
> Not sure if this would affect master, I'll add a test; ACID stats invalidation definitely
needs to invalidate all the stats, not just basic stats flag, when invalidating.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message