Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id E666A200B0F for ; Fri, 3 Jun 2016 05:07:53 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id E5258160A52; Fri, 3 Jun 2016 03:07:53 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 384CB160A51 for ; Fri, 3 Jun 2016 05:07:53 +0200 (CEST) Received: (qmail 48026 invoked by uid 500); 3 Jun 2016 03:07:52 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 48010 invoked by uid 99); 3 Jun 2016 03:07:51 -0000 Received: from reviews-vm.apache.org (HELO reviews.apache.org) (140.211.11.40) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 03 Jun 2016 03:07:51 +0000 Received: from reviews.apache.org (localhost [127.0.0.1]) by reviews.apache.org (Postfix) with ESMTP id 05EF42C651D; Fri, 3 Jun 2016 03:07:51 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============5632574345385261439==" MIME-Version: 1.0 Subject: Re: Review Request 48159: HIVE-13901: Hivemetastore add partitions can be slow depending on filesystems From: Rajesh Balamohan To: Ashutosh Chauhan Cc: hive , Rajesh Balamohan Date: Fri, 03 Jun 2016 03:07:51 -0000 Message-ID: <20160603030751.32447.86548@reviews.apache.org> X-ReviewBoard-URL: https://reviews.apache.org/ Auto-Submitted: auto-generated Sender: Rajesh Balamohan X-ReviewGroup: hive X-Auto-Response-Suppress: DR, RN, OOF, AutoReply X-ReviewRequest-URL: https://reviews.apache.org/r/48159/ X-Sender: Rajesh Balamohan References: <20160603030343.1106.22428@reviews.apache.org> In-Reply-To: <20160603030343.1106.22428@reviews.apache.org> Reply-To: Rajesh Balamohan X-ReviewRequest-Repository: hive-git archived-at: Fri, 03 Jun 2016 03:07:54 -0000 --===============5632574345385261439== MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48159/#review136027 ----------------------------------------------------------- metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java (line 2306) In case the cluster is configured with StorageBasedAuth*, based on the listener implementations, it can have similar issues where it ends up scanning all dirs for permissions etc. Similar approach can be followed there as well, but out of scope of this JIRA. Will create a separate jira for it. - Rajesh Balamohan On June 3, 2016, 3:03 a.m., Rajesh Balamohan wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48159/ > ----------------------------------------------------------- > > (Updated June 3, 2016, 3:03 a.m.) > > > Review request for hive and Ashutosh Chauhan. > > > Bugs: HIVE-13901 > https://issues.apache.org/jira/browse/HIVE-13901 > > > Repository: hive-git > > > Description > ------- > > Depending on FS, creating external tables & adding partitions can be expensive (e.g msck which adds all partitions). > > > Diffs > ----- > > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 9cc8fbe > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 94dd72e > > Diff: https://reviews.apache.org/r/48159/diff/ > > > Testing > ------- > > > Thanks, > > Rajesh Balamohan > > --===============5632574345385261439==--