hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-19820) add ACID stats support to background stats updater and fix bunch of edge cases found in SU tests
Date Wed, 11 Jul 2018 02:16:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-19820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539460#comment-16539460
] 

Sergey Shelukhin commented on HIVE-19820:
-----------------------------------------

Uploaded the full patch that makes sure every time txn stats are updated, we check that the
caller provides write ID list, write ID, etc. If the stats parameter is updated without those,
the call fails.
This required adding some more APIs based on request pattern, to pass the write ID/etc. to
calls that update column stats, and others, so the generated code patch is very large.

This allows analyze and other callers to correctly tell apart parallel callers that set stats
to invalid, vs stats that were invalid before analyze, since every caller must now set write
ID.
All the branch tests pass, and updater test now also has some test cases for running analyze
with parallel transactions and also aborted transactions, that also pass.
Lots of tests might fail because they will try to invalidate stats without write ID, I'll
look at them and fix those that fail. 
I'll probably add some more tests to updater test for truncate/etc., to validate updater and
also indirectly validate these commands.

cc [~steveyeom2017] [~ekoifman]

> add ACID stats support to background stats updater and fix bunch of edge cases found
in SU tests
> ------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-19820
>                 URL: https://issues.apache.org/jira/browse/HIVE-19820
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>            Priority: Major
>         Attachments: HIVE-19820.01-master-txnstats.patch, HIVE-19820.02-master-txnstats.patch,
HIVE-19820.03-master-txnstats.patch, HIVE-19820.04-master-txnstats.patch, HIVE-19820.patch,
branch-19820.nogen.patch
>
>
> Follow-up from HIVE-19418.
> Right now it checks whether stats are valid in an old-fashioned way... and also gets
ACID state, and discards it without using.
> When ACID stats are implemented, ACID state needs to be used to do version-aware valid
stats checks.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message