hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward Capriolo (JIRA)" <>
Subject [jira] Updated: (HIVE-716) Web Interface wait/notify, interface changes
Date Fri, 14 Aug 2009 14:40:14 GMT


Edward Capriolo updated HIVE-716:

    Status: Patch Available  (was: Open)

There were two main features I wanted to support in HWI.

1) The Query Box should allow multiple queries separated by ; This would make the application
more user friendly.

2) For queries with small result sets such as 'explain' or 'show tables' should be able possible
without dumping to a file

Wait/notify semantics

Loop w/sleep are ok but a Java threads support wait/notify so that is cleaner, nicer to code.
-               while (searchItem.getStatus() != HWISessionItem.WebSessionItemStatus.QUERY_COMPLETE)
-                       Thread.sleep(1);
+               synchronized (searchItem.runnable ) {
+                 while (searchItem.getStatus() != HWISessionItem.WebSessionItemStatus.READY)
+                       searchItem.runnable.wait();
+                  }

HWISessionItem Constructor
protected HWISessionItem ()

public HWISessionItem(HWIAuth auth, String sessionName) 

Having a public constrcutor is a plus the class could theoretically be used standalone.
The no argument constructor did not make sense, HWISessionItem was programmed originally like
a Java
Bean no arg, constructore, lots of setters/getters. Really to be instantaited this information
write once. It only makes sense to handle it in the constructor. (this also makes the lazy
initialization easier)

state machine
-       public enum WebSessionItemStatus {
-       };
+  /**  Represents the state a session item can be in. */
+  public enum WebSessionItemStatus {
+  };

THe state mechanics worked but the states were funky. QUERY_COMPLETE in particular bothered

NEW-Constructed but not initialized
READY-Initialized waiting for query set
QUERY-SET-User has signaled that the query should be ran
Query-running- the query is now running (dont change anything)  
back to ready READY
DESTROY-Fall out of run loop

Right now  KILL_QUERY is not terribly effective. With HiveHistory session can be killed from
the job tracker.

Removed private members w setters/getters
-       private SetProcessor sp;
-       private Driver qp;
These are not members they are only local to the runQuery() method

The framework now takes a list of queries and returns a list of results
+  private List<String>  queries;
+  queryRet = new ArrayList<Integer>();

Constructor blocks for lazy initiailization
There was a subtle bug in that the initialization of some variables happned in the thread
after the constructor returned. However a redirect on
the web site could send a user onto a page causing an exception. Now the constructor uses
wait/notify to properly wait for initialization
 l4j.debug("Wait for NEW->READY transition");
+    synchronized (this.runnable){
+      if (this.status != WebSessionItemStatus.READY) {
+       try {
+         this.runnable.wait();
+        } catch (Exception ex) {}
+      }
+    }
+    l4j.debug("NEW->READY transition complete");

run() uses state NEW not conf=null to detect when initialization is needed. Same effect more
self documenting

 while (this.status != HWISessionItem.WebSessionItemStatus.DESTROY) {

             if (conf == null) {
-          this.itemInit();
-                               }

 while (this.status != HWISessionItem.WebSessionItemStatus.DESTROY) {
+        if (this.status == WebSessionItemStatus.NEW) {
+         this.itemInit();

Feature Reuslt Bucket
a query like explain or show is not the type of thing you want to dump to a file just to look
at the file.
The resultBucket is a sized buffer that saves some query results to make them avaialable from
the web interface.
+            while (qp.getResults(res)) {
+             resultBucket.add(res);
+              if (resultBucket.size() > resultBucketMaxSize) 
+               resultBucket.remove(0);
+              for (String row : res) {
+               if (ss != null) { 
+                 if (ss.out != null) 
+                   ss.out.println(row);

Many changes went down here :) We could break this up into a few JIRA or bundle together.
The last round of changes were a good at cleanup up the SessionManager this round will have
the SessionItem nicely polished.

* Add ResultBucket buffer to web interface
* HWI to run multiple queries in a single run
* Changes to  HWI State Mechanics
* Web Interface format and license. 1 Propertly format and add apache licence to all HWI Source

Please review and lets talk about the best way to procede.


> Web Interface wait/notify, interface changes
> --------------------------------------------
>                 Key: HIVE-716
>                 URL:
>             Project: Hadoop Hive
>          Issue Type: Improvement
>          Components: Web UI
>         Environment: All
>            Reporter: Edward Capriolo
>            Assignee: Edward Capriolo
>         Attachments: hive-716-2.diff, hive-716-3.diff, hive-716.diff, hwi_query_box.png
> In TestHWISessionItem 
> Asserts are backwards
> {noformat}
> assertEquals(  searchItem.getQueryRet(), 0);
> {noformat}
> Should be
> {noformat}
> assertEquals( zero , searchItem.getQueryRet());
> {noformat}
> Wait/notify semantics can be added. This is helpful for end user, and cleaner in the
test case.
> {noformat}
> while (user1_item2.getStatus() != HWISessionItem.WebSessionItemStatus.QUERY_COMPLETE)
>   Thread.sleep(1);
> }
> {noformat}
> {noformat}
> synchronized (user1_item2.runnable) {
>   while (user1_item2.getStatus() != HWISessionItem.WebSessionItemStatus.QUERY_COMPLETE)
>      user1_item2.runnable.wait();
>   }
> }
> {noformat}
> The text box in the web interface should accept multiple queries separated by ';' like
the cli does. This will add more usability. No need for separate set processor pages. 
> setQuery(String) is replaced by setQueries(List<String>)

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message