From reviews-return-1015746-archive-asf-public=cust-asf.ponee.io@spark.apache.org Fri Jan 17 02:25:29 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id B418218060E for ; Fri, 17 Jan 2020 03:25:28 +0100 (CET) Received: (qmail 92108 invoked by uid 500); 17 Jan 2020 02:25:28 -0000 Mailing-List: contact reviews-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list reviews@spark.apache.org Received: (qmail 92096 invoked by uid 99); 17 Jan 2020 02:25:28 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 17 Jan 2020 02:25:28 +0000 From: GitBox To: reviews@spark.apache.org Subject: [GitHub] [spark] AngersZhuuuu edited a comment on issue #27232: [SPARK-30525][SQL]HiveTableScanExec do not need to prune partitions again after pushing down to hive metastore Message-ID: <157922792803.2217.11265649322726194113.gitbox@gitbox.apache.org> References: In-Reply-To: Date: Fri, 17 Jan 2020 02:25:28 -0000 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit AngersZhuuuu edited a comment on issue #27232: [SPARK-30525][SQL]HiveTableScanExec do not need to prune partitions again after pushing down to hive metastore URL: https://github.com/apache/spark/pull/27232#issuecomment-575435484 emmmm, prune partition again is because when call `listPartitionsByFilter` kit can't convert all spark filter expression to hive metastore filter condition. and will get more partition then exactly wanted. then prune again will spark's method to keep smallest partitions. If we can't promise all these case was fixed in `listPartitionsByFilter`, we may still need this. and it's cost is negligible ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org For additional commands, e-mail: reviews-help@spark.apache.org