hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kevin Wilfong" <>
Subject Review Request: Need a way to categorize queries in hooks for improved logging
Date Fri, 16 Sep 2011 19:04:32 GMT

This is an automatically generated e-mail. To reply, visit:

Review request for hive and Ning Zhang.


The information that would be useful for categorizing queries is clearest in the Semantic
Analyzer, when the data from the Parser is interpreted.  I added a new class which is designed
to collect that data here, and place it ultimately in the QueryPlan where it will be available
to hooks.

The information I collect is whether or not the query has the following clauses:
  Group By
  Order By
  Sort By
  Group By after a Join clause

Also, I store whether or not a script is used for mapping or reducing.

This addresses bug HIVE-2453.


  trunk/ql/src/java/org/apache/hadoop/hive/ql/ 1170719 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/ PRE-CREATION 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ 1170719 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ 1170719 
  trunk/ql/src/test/org/apache/hadoop/hive/ql/hooks/ PRE-CREATION

  trunk/ql/src/test/queries/clientpositive/query_properties.q PRE-CREATION 
  trunk/ql/src/test/results/clientpositive/query_properties.q.out PRE-CREATION 



I added a new test, which runs a variety of queries, such that each of the flags in QueryProperties
is set by at least one query, and also some are set in combinations.
I also added a hook which prints the contents of QueryProperties to error on the console.

I checked the output in the results file and verified it matched what I expected.



  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message