hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Szehon Ho <>
Subject Re: Review Request 43008: HIVE-12952 : Show query sub-pages on webui
Date Mon, 01 Feb 2016 20:05:29 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated Feb. 1, 2016, 8:05 p.m.)

Review request for hive.


Address Lenni review comments.

Also make a bug fix where compile-time PerfLogger was not getting displayed (it is on a different
thread, so added a checkpoint at end of compile-phase to save it)

Bugs: HIVE-12952

Repository: hive-git


This patch shows a query sub-page on WebUI, with detailed information of query on differnt

1.  Tabl- Base Info, ie user, query string, query id, begin time, end time, execution engine,
error (if any)
2.  Tab2- Query Plan
3.  Tab3- Stages (MR jobs), their progress and info
4.  Tab4- Call trace info captured from HMSClient and PerfLogger.

Implementation notes:
The UI design choices are inspired from Impala, and HBase.  This, like HBase webui, uses Jamon,
which is a superset of JSP and makes dynamic content a lot easier.  As such, brought in jamon
dependency and also js bootstrap libraries to support the dynamic tabs.

On Hive side, refactored webui query logic into following classes:  SQLoperationDisplay (info
captured from SQLOperation), QueryDisplay (info captured from Driver).

1. Hard to get more MR job information for the stages including a job-tracking url, due to
MR JobSubmission being a separate process, need to think about it.  Same for Spark/tez.
2. The explain plan might be a bit bulky and consume a bit of memory (though can tune with
"hive.server2.webui.max.historic.queries").  Perhaps in future we can spill to local disk,
and stream from there.  This might also help with (1), if we don't want to implement inter-process

Diffs (updated)

  common/src/java/org/apache/hadoop/hive/conf/ ff376a8 
  common/src/java/org/apache/hadoop/hive/ql/log/ d4194cf 
  common/src/java/org/apache/hive/http/ 9e23b11 
  pom.xml 802d3d4 
  ql/src/java/org/apache/hadoop/hive/ql/ 4c89812 
  ql/src/java/org/apache/hadoop/hive/ql/ PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/ 0bab769 
  service/pom.xml b2e3a84 
  service/src/jamon/org/apache/hive/tmpl/QueryProfileTmpl.jamon PRE-CREATION 
  service/src/java/org/apache/hive/service/cli/operation/ 0c263cf 
  service/src/java/org/apache/hive/service/cli/operation/ f1ce6f6 
  service/src/java/org/apache/hive/service/cli/operation/ 01b1d3d 
  service/src/java/org/apache/hive/service/cli/operation/ PRE-CREATION

  service/src/java/org/apache/hive/service/cli/operation/ PRE-CREATION

  service/src/java/org/apache/hive/service/cli/operation/ 179f6dd 
  service/src/java/org/apache/hive/service/server/ 958458f 
  service/src/java/org/apache/hive/service/servlet/ PRE-CREATION 
  service/src/resources/hive-webapps/hiveserver2/hiveserver2.jsp a0b5d2e 
  service/src/resources/hive-webapps/static/js/bootstrap.js PRE-CREATION 
  service/src/resources/hive-webapps/static/js/bootstrap.min.js PRE-CREATION 
  service/src/resources/hive-webapps/static/js/jquery.min.js PRE-CREATION 
  service/src/resources/hive-webapps/static/js/tab.js PRE-CREATION 



Manual testing.  Can add some unit tests in follow-up.


Szehon Ho

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message