From issues-return-111271-archive-asf-public=cust-asf.ponee.io@hive.apache.org Mon Apr 2 23:41:04 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id DE07118067C for ; Mon, 2 Apr 2018 23:41:03 +0200 (CEST) Received: (qmail 22644 invoked by uid 500); 2 Apr 2018 21:41:03 -0000 Mailing-List: contact issues-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list issues@hive.apache.org Received: (qmail 22590 invoked by uid 99); 2 Apr 2018 21:41:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Apr 2018 21:41:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id A9D551804A8 for ; Mon, 2 Apr 2018 21:41:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.511 X-Spam-Level: X-Spam-Status: No, score=-109.511 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id qXsyobYDfo6E for ; Mon, 2 Apr 2018 21:41:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 6764C5FB37 for ; Mon, 2 Apr 2018 21:41:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 844A5E0354 for ; Mon, 2 Apr 2018 21:41:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 2FC8D2560E for ; Mon, 2 Apr 2018 21:41:00 +0000 (UTC) Date: Mon, 2 Apr 2018 21:41:00 +0000 (UTC) From: "Janaki Lahorani (JIRA)" To: issues@hive.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HIVE-17824) msck repair table should drop the missing partitions from metastore MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-17824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janaki Lahorani updated HIVE-17824: ----------------------------------- Attachment: HIVE-17824.1.patch > msck repair table should drop the missing partitions from metastore > ------------------------------------------------------------------- > > Key: HIVE-17824 > URL: https://issues.apache.org/jira/browse/HIVE-17824 > Project: Hive > Issue Type: Improvement > Reporter: Vihang Karajgaonkar > Assignee: Janaki Lahorani > Priority: Major > Attachments: HIVE-17824.1.patch > > > {{msck repair table }} is often used in environments where the new partitions are loaded as directories on HDFS or S3 and users want to create the missing partitions in bulk. However, currently it only supports addition of missing partitions. If there are any partitions which are present in metastore but not on the FileSystem, it should also delete them so that it truly repairs the table metadata. > We should be careful not to break backwards compatibility so we should either introduce a new config or keyword to add support to delete unnecessary partitions from the metastore. This way users who want the old behavior can easily turn it off. -- This message was sent by Atlassian JIRA (v7.6.3#76005)