hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kevin Wilfong" <kevinwilf...@fb.com>
Subject Review Request: Add StatsTask to task tree for CTAS query.
Date Thu, 29 Sep 2011 21:48:03 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2120/
-----------------------------------------------------------

Review request for hive, Yongqiang He and Ning Zhang.


Summary
-------

(This is not needed for RC-2)

I modified StatsTask so that it could be created from a CTAS query.

The StatsTask is created where StatsTask are typically created, but instead of being appended
to the final MoveTask it is stored in the QB.  This is because the SemanticAnalyzer appends
the task to create the table to the task tree in order to ensure atomicity.  The StatsTask
requires that the table already be created, by putting it in the QB, the SemanticAnalyzer
is able to access it and append to it to the create table task.


This addresses bug HIVE-2472.
    https://issues.apache.org/jira/browse/HIVE-2472


Diffs
-----

  trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/StatsTask.java 1177363 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRFileSink1.java 1177363 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/QB.java 1177363 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1177363 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/StatsWork.java 1177363 
  trunk/ql/src/test/results/clientpositive/ctas.q.out 1177363 
  trunk/ql/src/test/results/clientpositive/merge3.q.out 1177363 
  trunk/ql/src/test/results/clientpositive/rcfile_createas1.q.out 1177363 
  trunk/ql/src/test/results/clientpositive/smb_mapjoin9.q.out 1177363 

Diff: https://reviews.apache.org/r/2120/diff


Testing
-------

I ran a CTAS query and verified the stats appeared in the console at the end of the query,
and that they were stored in the table's metadata.

I ran the unit test queries, and updated the output of the ones which use CTAS queries


Thanks,

Kevin


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message