hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward Capriolo (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-278) Add HiveHistory to Hive web interface
Date Thu, 19 Mar 2009 01:07:50 GMT

    [ https://issues.apache.org/jira/browse/HIVE-278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12683266#action_12683266
] 

Edward Capriolo commented on HIVE-278:
--------------------------------------

1. Why remove the getJobTrackerURI() and getKillCommand() functions?
These were never reachable from the web interface. I notice that the HiveHistory is written
structured but reading it results in string value pairs. The orignal goal of this method was
to return a URL to this job on the jobtracker. Now its seems like there are multiple URLs
the completed and the current. This method might be misnamed.  Should we keep them in and
unimplemented?

I can explain the answer to 2 and 3 in a use case of the old and the new.

Use Case 1 OLD FLOW: User starts a session and runs a query.
1) HWISessionManager.createSession is sent the HWIAuth (username and password) as well as
a session name.
2) HWISessionManager creates an HWISessionItem.
3) HWISessionManager then needed to call itemInit() for step 4
4) HWISessionManager then used the si.setProcessorQuery() to set the auth information

User now sets uses the web interface to set the query up.
Finally the user sets calls clientStart() the session items status is set to QUERY_SET
5) The HWISessionManager is a thread that wakes up and iterates a List looking for QUERY_SET
6) if QUERY_SET call HWISessionItem.startIt()
7) startIt() created a thread
8) the thread ran the query to completion then ended. Basically a one shot thread.

The problem with the Hive History highlighted this one shot thread problem. The nature of
servlet threading posed its challenges. HWISessionManager and HWISessionItem posed its own
set of challenges.

The troubleshooting you guys did concluded that some code would have to be moved around. After
looking at the code I decided that steps 3 and 4 could be moved to HWISessionItem. Once steps
3 and 4 are moved HWISessionManager, you realize steps 5 6 7 are a two thread interaction
and it can be done with one.

Use Case 2 NEW FLOW: User starts a session and runs a query
1) HWISessionManager.createSession is sent the HWIAuth (username and password) as well as
a session name.
2) HWIAuth is passed to HWISessionItem HWISessionItem.status=NEW
3) HWISessionItem enters its run()

User now sets uses the web interface to set the query up.
4) HWISessionItem.status changes to QUERY_SET. notify() is called

5) Thread will now wake up
6) conf will be null so itemInit() will be called
7) the status is QUERY_SET this.runQuery() is called status is now QUERY_RUNNING
8) The query completes the status is now QUERY_COMPLETE

Moving more code to HWISessionItem cuts down on the back and forth between the two classes.

So the logic of run()
if (status = DESTROY)
fall out of the thread loop.
if (status = QUERY_SET)
start the query
if (conf = null)
itemInit()
else wait() --user notify() will eventually wake this thread up

> Add HiveHistory to Hive web interface
> -------------------------------------
>
>                 Key: HIVE-278
>                 URL: https://issues.apache.org/jira/browse/HIVE-278
>             Project: Hadoop Hive
>          Issue Type: New Feature
>          Components: Logging, Web UI
>    Affects Versions: 0.3.0
>            Reporter: Edward Capriolo
>            Assignee: Edward Capriolo
>            Priority: Minor
>             Fix For: 0.3.0
>
>         Attachments: HIVE-278-1.diff, HIVE-278.diff, hive_history.png, session_logging.diff
>
>
> In order for HIVE-176 to be utilized by the Hive web interface a few changes need to
be made.
> * HWISessionItem needs a method with an argument signature 
> {noformat} 
> public HiveHistoryViewer getHistoryViewer() throws HWIException
> {noformat} 
> * session_manage.jsp needs an addition
> {noformat} 
>  Hive History: <a href=/hwi/session_history.jsp?sessionName=<%=sessionName%>"><%=sessionName%></a><br>
> {noformat} 
> * session_history.jsp will have to be created to use the ql.history api

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message