hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasanth Jayachandran (JIRA)" <>
Subject [jira] [Updated] (HIVE-15250) Reuse partitions info generated in MoveTask to its subscribers (StatsTask)
Date Wed, 30 Nov 2016 06:02:58 GMT


Prasanth Jayachandran updated HIVE-15250:
          Resolution: Fixed
       Fix Version/s: 2.2.0
    Target Version/s: 2.2.0
              Status: Resolved  (was: Patch Available)

Committed to master. Thanks [~rajesh.balamohan] for the patch!

> Reuse partitions info generated in MoveTask to its subscribers (StatsTask)          
> -----------------------------------------------------------------------------------------
>                 Key: HIVE-15250
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>          Components: Metastore
>    Affects Versions: 2.2.0
>            Reporter: Rajesh Balamohan
>            Assignee: Rajesh Balamohan
>            Priority: Minor
>             Fix For: 2.2.0
>         Attachments: HIVE-15250.1.patch, HIVE-15250.2.patch, HIVE-15250.3.patch
> When dynamic partitions are enabled, {{StatsTask}} loads partition information by querying
metastore. In cases like {{insert overwrite table}}, this can be expensive operation depending
on the number of partitions involved (for e.g, in tpcds populating web_returns table would
incur 2184 DB calls just on this function).
> It would be good to pass on the partition information generated in MoveTask to its subscribers
to reduce the number of DB calls.

This message was sent by Atlassian JIRA

View raw message