hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chinna Rao Lalam (JIRA)" <>
Subject [jira] [Updated] (HIVE-12077) MSCK Repair table should fix partitions in batches
Date Tue, 26 Jul 2016 08:39:20 GMT


Chinna Rao Lalam updated HIVE-12077:
    Attachment: HIVE-12077.5.patch

> MSCK Repair table should fix partitions in batches 
> ---------------------------------------------------
>                 Key: HIVE-12077
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: Hive
>            Reporter: Ryan P
>            Assignee: Chinna Rao Lalam
>         Attachments: HIVE-12077.1.patch, HIVE-12077.2.patch, HIVE-12077.3.patch, HIVE-12077.4.patch,
> If a user attempts to run MSCK REPAIR TABLE on a directory with a large number of untracked
partitions HMS will OOME. I suspect this is because it attempts to do one large bulk load
in an effort to save time. Ultimately this can lead to a collection so large in size that
HMS eventually hits an Out of Memory Exception. 
> Instead I suggest that Hive include a configurable batch size that HMS can use to break
up the load. 

This message was sent by Atlassian JIRA

View raw message