Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D999811213 for ; Wed, 13 Aug 2014 04:51:22 +0000 (UTC) Received: (qmail 85944 invoked by uid 500); 13 Aug 2014 04:51:22 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 85868 invoked by uid 500); 13 Aug 2014 04:51:22 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 85849 invoked by uid 99); 13 Aug 2014 04:51:22 -0000 Received: from reviews-vm.apache.org (HELO reviews.apache.org) (140.211.11.40) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 13 Aug 2014 04:51:22 +0000 Received: from reviews.apache.org (localhost [127.0.0.1]) by reviews.apache.org (Postfix) with ESMTP id C42231D9D0C; Wed, 13 Aug 2014 04:51:21 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============5479807785517873061==" MIME-Version: 1.0 Subject: Re: Review Request 24289: MetadataUpdater: provide a mechanism to edit the statistics of a column in a table (or a partition of a table) From: "pengcheng xiong" To: "Lars Francke" , "pengcheng xiong" , "hive" , "Ashutosh Chauhan" Date: Wed, 13 Aug 2014 04:51:21 -0000 Message-ID: <20140813045121.1261.55855@reviews.apache.org> X-ReviewBoard-URL: https://reviews.apache.org Auto-Submitted: auto-generated Sender: "pengcheng xiong" X-ReviewGroup: hive X-ReviewRequest-URL: https://reviews.apache.org/r/24289/ X-Sender: "pengcheng xiong" References: <20140812182347.1261.69798@reviews.apache.org> In-Reply-To: <20140812182347.1261.69798@reviews.apache.org> Reply-To: "pengcheng xiong" X-ReviewRequest-Repository: hive-git --===============5479807785517873061== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24289/ ----------------------------------------------------------- (Updated Aug. 13, 2014, 4:51 a.m.) Review request for hive. Changes ------- add Privilege Repository: hive-git Description ------- This patch provides ability to update certain stats without scanning any data or without "hacking the backend db". It helps (esp for CBO work) to set up unit tests quickly and verify both cbo and the stats subsystem. It also helps when experimenting with the system if you're just trying out hive/hadoop on a small cluster. Finally it gives you a quick and clean way to fix things when something went wrong wrt stats in your environment. Usage: ALTER TABLE table_name PARTITION partition_spec UPDATE STATISTICS FOR COLUMN col_name SET col_statistics For example, ALTER TABLE src_x_int UPDATE STATISTICS FOR COLUMN key SET ('numDVs'='101','highValue'='10001.0'); ALTER TABLE src_p PARTITION(partitionId=1) UPDATE STATISTICS FOR COLUMN key SET ('numDVs'='100','avgColLen'='1.0001'); Diffs (updated) ----- metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java a5f40b1 metastore/src/model/org/apache/hadoop/hive/metastore/model/MPartitionColumnStatistics.java 89c31dc metastore/src/model/org/apache/hadoop/hive/metastore/model/MTableColumnStatistics.java 44bbab5 ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnStatsUpdateTask.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/TaskFactory.java 24dfed1 ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 0e705aa ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g bf77d1c ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 268920a ql/src/java/org/apache/hadoop/hive/ql/plan/ColumnStatsUpdateWork.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java 6a447ea ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveOperationType.java 81366e3 ql/src/test/queries/clientpositive/alter_partition_update_status.q PRE-CREATION ql/src/test/queries/clientpositive/alter_table_update_status.q PRE-CREATION ql/src/test/results/clientpositive/alter_partition_update_status.q.out PRE-CREATION ql/src/test/results/clientpositive/alter_table_update_status.q.out PRE-CREATION Diff: https://reviews.apache.org/r/24289/diff/ Testing ------- File Attachments ---------------- HIVE-7506.5.patch https://reviews.apache.org/media/uploaded/files/2014/08/12/db6d16c3-a938-4995-9ae4-401eaae3b676__HIVE-7506.5.patch Thanks, pengcheng xiong --===============5479807785517873061==--