hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vihang Karajgaonkar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-19344) Change default value of msck.repair.batch.size
Date Sat, 05 May 2018 01:35:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-19344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16464556#comment-16464556
] 

Vihang Karajgaonkar commented on HIVE-19344:
--------------------------------------------

looks like ptest skipped this patch. 

> Change default value of msck.repair.batch.size 
> -----------------------------------------------
>
>                 Key: HIVE-19344
>                 URL: https://issues.apache.org/jira/browse/HIVE-19344
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Vihang Karajgaonkar
>            Assignee: Vihang Karajgaonkar
>            Priority: Minor
>         Attachments: HIVE-19344.01.patch, HIVE-19344.02.patch
>
>
> {{msck.repair.batch.size}} default to 0 which means msck will try to add all the partitions
in one API call to HMS. This can potentially add huge memory pressure on HMS. The default
value should be changed to a reasonable number so that in case of large number of partitions
we can batch the addition of partitions. Same goes for {{msck.repair.batch.max.retries}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message