hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward Capriolo (JIRA)" <>
Subject [jira] Commented: (HIVE-713) Integrate operator counters into query plan to view progress of queries
Date Thu, 06 Aug 2009 19:15:14 GMT


Edward Capriolo commented on HIVE-713:

True. If we were able to implement JMX we could attach to a mapper/reduce and profile it directly.
IE real time how much memory a mapper is using, there are a bunch of standard jmx objects
that could be useful. Possibly a logging based approach might be a better way to go. Another
technique might be a job completion URL. counters+job completion would be a nice combination.

JMX could be a tricky implementation due to the multi-jvm nature of the hadoop. But if the
framework is managing counters for us getting them into JMX should be doable. I will review
your code more.

> Integrate operator counters into query plan to view progress of queries
> -----------------------------------------------------------------------
>                 Key: HIVE-713
>                 URL:
>             Project: Hadoop Hive
>          Issue Type: New Feature
>          Components: Logging
>            Reporter: Raghotham Murthy
>            Assignee: Raghotham Murthy
>         Attachments: hive-713.1.donot.patch, hive-713.2.donot.patch, hive-713.2.patch,
hive-713.3.patch, hive-713.4.donot.patch, hive-713.5.donot.patch, hive.713.1.good.patch, hive.713.patch.bad
> Annotate the QueryPlan - with multiple stages and each stage(Task) can have an Operator
tree - with counters that each Operator/Task updates.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message