hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joydeep Sen Sarma (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-176) structured log for obtaining query stats/info
Date Tue, 30 Dec 2008 19:23:44 GMT

    [ https://issues.apache.org/jira/browse/HIVE-176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12659901#action_12659901

Joydeep Sen Sarma commented on HIVE-176:

still reviewing this.

one problem is that the HiveHistory is not thread safe (and this matters for multithreaded
thrift/web servers). It looks like there is one history per session - right? In which case
- can you just put a hivehistory object as part of sessionstate? Sessionstate captures all
the session specific stuff - and is maintained thread local. when a new sessionstate is created
- the hivehistory can be created within it.

also - a little uncomfortable with the fixed /tmp setting for the history file. It would be
good to have a new option in HiveConf for the history file directory. This would something
that admins would want to tweak for the hive server deployments (hadoop has a similar one)
and users would want to tweak to get all their history files in a standard location. (The
default  could be same as SCRATCHDIR that's already defined in HiveConf.

will send comments on the rest of the stuff asap.

> structured log for obtaining query stats/info
> ---------------------------------------------
>                 Key: HIVE-176
>                 URL: https://issues.apache.org/jira/browse/HIVE-176
>             Project: Hadoop Hive
>          Issue Type: Bug
>          Components: Logging
>    Affects Versions: 0.2.0
>            Reporter: Joydeep Sen Sarma
>            Assignee: Suresh Antony
>             Fix For: 0.2.0
>         Attachments: patch_176.txt
> Josh <josh@besquared.net> wrote:
> When launching off hive queries using hive -e is there a way to get the job id so that
I can just queue them up and go check their statuses later? What's the general pattern for
queueing and monitoring without using the libraries directly?
> I'm gonna throw my vote in for a structured log format. Users could tail it and use whatever
queuing or monitoring they wish. It's also probably just a 30 minute project for someone already
familiar with the code. I suggest ^A seperated key=value pairs per log line.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message