hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward Capriolo (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-713) Integrate operator counters into query plan to view progress of queries
Date Thu, 06 Aug 2009 18:21:14 GMT

    [ https://issues.apache.org/jira/browse/HIVE-713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12740182#action_12740182

Edward Capriolo commented on HIVE-713:

I am looking to get some hive-jmx integeration (HIVE-551). One problem I ran into was dealing
with true distributed operation. I am wondered where the overlap in what we are doing is and
or how you are tackling this.

1. For hive 551 I am thinking each map/reduce can open a separate JMX port. This is a problem
because first multiple mappers might run on the same machine. Secondly, I think i would have
to write the location jmx ports to /tmp space so someone can read the file and figure out
what to attach to.

My thinking use JMX proxy. We know that each system is running a task-tracker. From a mapper/reduce
we can proxy the task tracker and add MX beans named with query-id. I wonder can we get counters/jmx
working together somehow? 

> Integrate operator counters into query plan to view progress of queries
> -----------------------------------------------------------------------
>                 Key: HIVE-713
>                 URL: https://issues.apache.org/jira/browse/HIVE-713
>             Project: Hadoop Hive
>          Issue Type: New Feature
>          Components: Logging
>            Reporter: Raghotham Murthy
>            Assignee: Raghotham Murthy
>         Attachments: hive-713.1.donot.patch, hive-713.2.donot.patch, hive-713.2.patch,
hive-713.3.patch, hive-713.4.donot.patch, hive-713.5.donot.patch, hive.713.1.good.patch, hive.713.patch.bad
> Annotate the QueryPlan - with multiple stages and each stage(Task) can have an Operator
tree - with counters that each Operator/Task updates.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message