hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-679) XML-based metrics as JSP servlet for JobTracker
Date Sat, 29 Aug 2009 00:52:33 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749048#action_12749048

Chris Douglas commented on MAPREDUCE-679:

bq. Chris, to what API are you referring?

o.a.h.metrics and o.a.h.mapred.JobTrackerInstrumentation (JobTrackerMetricsInst). Ganglia,
Chukwa, etc. push metrics through this interface. Adding a new interface isn't a small change.
For example, the current patch calls {{JobTracker::runningJobs}} rather than {{JobTracker::getRunningJobs}};
the former is unsynchronized and accesses shared data structures. There have been similar
issues caused by metrics frameworks attempting to pull information out of the JobTracker without
an audit of the (regrettable) lock hierarchy. If this receives its data through the metrics
API, then neither you nor maintainers need to consider how this servlet affects the shared
JT data.

bq. No API reports job history in a universally-readable manner (e.g., REST + XML).

That's fair. Again, I like machine-readable formats and wouldn't object to this even if it
were an interim solution, but I want to be clear about how it gets its data and what it supports,
since we'll be committing ourselves to maintaining both.

Would JobHistory be a better home for this? Much of the data are not currently available in
the web UI, let alone in a reasonable format.

> XML-based metrics as JSP servlet for JobTracker
> -----------------------------------------------
>                 Key: MAPREDUCE-679
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-679
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: jobtracker
>            Reporter: Aaron Kimball
>            Assignee: Aaron Kimball
>         Attachments: example-jobtracker-completed-job.xml, example-jobtracker-running-job.xml,
MAPREDUCE-679.2.patch, MAPREDUCE-679.3.patch, MAPREDUCE-679.4.patch, MAPREDUCE-679.patch
> In HADOOP-4559, a general REST API for reporting metrics was proposed but work seems
to have stalled. In the interim, we have a simple XML translation of the existing JobTracker
status page which provides the same metrics (including the tables of running/completed/failed
jobs) as the human-readable page. This is a relatively lightweight addition to provide some
machine-understandable metrics reporting.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message