Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 891DA200C77 for ; Mon, 1 May 2017 18:49:58 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 87A6B160BAE; Mon, 1 May 2017 16:49:58 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id CF615160BA0 for ; Mon, 1 May 2017 18:49:57 +0200 (CEST) Received: (qmail 92020 invoked by uid 500); 1 May 2017 16:49:56 -0000 Mailing-List: contact commits-help@hawq.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hawq.incubator.apache.org Delivered-To: mailing list commits@hawq.incubator.apache.org Received: (qmail 92010 invoked by uid 99); 1 May 2017 16:49:56 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 May 2017 16:49:56 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id BAFFAC047C for ; Mon, 1 May 2017 16:49:55 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -4.222 X-Spam-Level: X-Spam-Status: No, score=-4.222 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id qqtH1UaGjjUr for ; Mon, 1 May 2017 16:49:54 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with SMTP id 3A8775FDF4 for ; Mon, 1 May 2017 16:49:54 +0000 (UTC) Received: (qmail 92007 invoked by uid 99); 1 May 2017 16:49:53 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 May 2017 16:49:53 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 462BDDFB91; Mon, 1 May 2017 16:49:53 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: kavinder@apache.org To: commits@hawq.incubator.apache.org Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: incubator-hawq git commit: HAWQ-1440. Support ANALYZE for all Hive External Tables Date: Mon, 1 May 2017 16:49:53 +0000 (UTC) archived-at: Mon, 01 May 2017 16:49:58 -0000 Repository: incubator-hawq Updated Branches: refs/heads/master e082d3adb -> 0a754883f HAWQ-1440. Support ANALYZE for all Hive External Tables Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/0a754883 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/0a754883 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/0a754883 Branch: refs/heads/master Commit: 0a754883f2b9c02e6c85a50e1aa82d853791d09e Parents: e082d3a Author: Kavinder Dhaliwal Authored: Tue Apr 25 15:14:47 2017 -0700 Committer: Kavinder Dhaliwal Committed: Mon May 1 09:46:44 2017 -0700 ---------------------------------------------------------------------- .../apache/hawq/pxf/plugins/hive/HiveDataFragmenter.java | 11 +++++++++-- .../hawq/pxf/plugins/hive/HiveInputFormatFragmenter.java | 2 +- 2 files changed, 10 insertions(+), 3 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/0a754883/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/HiveDataFragmenter.java ---------------------------------------------------------------------- diff --git a/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/HiveDataFragmenter.java b/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/HiveDataFragmenter.java index a03d3b7..9cf8f27 100644 --- a/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/HiveDataFragmenter.java +++ b/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/HiveDataFragmenter.java @@ -466,7 +466,14 @@ public class HiveDataFragmenter extends Fragmenter { */ @Override public FragmentsStats getFragmentsStats() throws Exception { - throw new UnsupportedOperationException( - "ANALYZE for Hive plugin is not supported"); + Metadata.Item tblDesc = HiveUtilities.extractTableFromName(inputData.getDataSource()); + Table tbl = HiveUtilities.getHiveTable(client, tblDesc); + Metadata metadata = new Metadata(tblDesc); + HiveUtilities.getSchema(tbl, metadata); + + long split_count = Long.parseLong(tbl.getParameters().get("numFiles")); + long totalSize = Long.parseLong(tbl.getParameters().get("totalSize")); + long firstFragmentSize = totalSize / split_count; + return new FragmentsStats(split_count, firstFragmentSize, totalSize); } } http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/0a754883/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/HiveInputFormatFragmenter.java ---------------------------------------------------------------------- diff --git a/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/HiveInputFormatFragmenter.java b/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/HiveInputFormatFragmenter.java index 9199118..b133e1d 100644 --- a/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/HiveInputFormatFragmenter.java +++ b/pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/HiveInputFormatFragmenter.java @@ -121,6 +121,6 @@ public class HiveInputFormatFragmenter extends HiveDataFragmenter { */ @Override public FragmentsStats getFragmentsStats() throws Exception { - throw new UnsupportedOperationException("ANALYZE for HiveRc, HiveText, and HiveOrc plugins is not supported"); + return super.getFragmentsStats(); } }