From issues-return-116350-archive-asf-public=cust-asf.ponee.io@hive.apache.org Thu May 3 19:36:06 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 83512180675 for ; Thu, 3 May 2018 19:36:05 +0200 (CEST) Received: (qmail 31065 invoked by uid 500); 3 May 2018 17:36:04 -0000 Mailing-List: contact issues-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list issues@hive.apache.org Received: (qmail 31056 invoked by uid 99); 3 May 2018 17:36:04 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 May 2018 17:36:04 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 12D39C8814 for ; Thu, 3 May 2018 17:36:04 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -110.301 X-Spam-Level: X-Spam-Status: No, score=-110.301 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id CrTWa44udBRP for ; Thu, 3 May 2018 17:36:03 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 9A4255FC6F for ; Thu, 3 May 2018 17:36:02 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 53405E12C6 for ; Thu, 3 May 2018 17:36:02 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id EEA01212A9 for ; Thu, 3 May 2018 17:36:00 +0000 (UTC) Date: Thu, 3 May 2018 17:36:00 +0000 (UTC) From: "Vihang Karajgaonkar (JIRA)" To: issues@hive.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HIVE-19344) Change default value of msck.repair.batch.size MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-19344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-19344: --------------------------------------- Attachment: HIVE-19344.01.patch > Change default value of msck.repair.batch.size > ----------------------------------------------- > > Key: HIVE-19344 > URL: https://issues.apache.org/jira/browse/HIVE-19344 > Project: Hive > Issue Type: Improvement > Reporter: Vihang Karajgaonkar > Assignee: Vihang Karajgaonkar > Priority: Minor > Attachments: HIVE-19344.01.patch > > > {{msck.repair.batch.size}} default to 0 which means msck will try to add all the partitions in one API call to HMS. This can potentially add huge memory pressure on HMS. The default value should be changed to a reasonable number so that in case of large number of partitions we can batch the addition of partitions. Same goes for {{msck.repair.batch.max.retries}} -- This message was sent by Atlassian JIRA (v7.6.3#76005)