hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "pengcheng xiong" <pxi...@hortonworks.com>
Subject Re: Review Request 24289: MetadataUpdater: provide a mechanism to edit the statistics of a column in a table (or a partition of a table)
Date Tue, 12 Aug 2014 18:00:09 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24289/
-----------------------------------------------------------

(Updated Aug. 12, 2014, 6 p.m.)


Review request for hive.


Changes
-------

add ALTERTABLE_UPDATETABLESTATS and ALTERTABLE_UPDATEPARTSTATS to HiveOperationType


Repository: hive-git


Description
-------

This patch provides ability to update certain stats without scanning any data or without "hacking
the backend db". It helps (esp for CBO work) to set up unit tests quickly and verify both
cbo and the stats subsystem. It also helps when experimenting with the system if you're just
trying out hive/hadoop on a small cluster. Finally it gives you a quick and clean way to fix
things when something went wrong wrt stats in your environment.
Usage:
ALTER TABLE table_name PARTITION partition_spec UPDATE STATISTICS FOR COLUMN col_name SET
col_statistics
For example,
ALTER TABLE src_x_int UPDATE STATISTICS FOR COLUMN key SET ('numDVs'='101','highValue'='10001.0');
ALTER TABLE src_p PARTITION(partitionId=1) UPDATE STATISTICS FOR COLUMN key SET ('numDVs'='100','avgColLen'='1.0001');


Diffs (updated)
-----

  metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java a5f40b1 
  metastore/src/model/org/apache/hadoop/hive/metastore/model/MPartitionColumnStatistics.java
89c31dc 
  metastore/src/model/org/apache/hadoop/hive/metastore/model/MTableColumnStatistics.java 44bbab5

  ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnStatsUpdateTask.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/TaskFactory.java 24dfed1 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 67a3aa7 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g ab1188a 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 268920a 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ColumnStatsUpdateWork.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java 6a447ea 
  ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/HiveOperationType.java
81366e3 
  ql/src/test/queries/clientpositive/alter_partition_update_status.q PRE-CREATION 
  ql/src/test/queries/clientpositive/alter_table_update_status.q PRE-CREATION 
  ql/src/test/results/clientpositive/alter_partition_update_status.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/alter_table_update_status.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/24289/diff/


Testing
-------


File Attachments
----------------

HIVE-7506.5.patch
  https://reviews.apache.org/media/uploaded/files/2014/08/12/db6d16c3-a938-4995-9ae4-401eaae3b676__HIVE-7506.5.patch


Thanks,

pengcheng xiong


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message