hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sreekanth Ramakrishnan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1219) JobTracker Metrics causes undue load on JobTracker
Date Tue, 20 Apr 2010 09:40:11 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858753#action_12858753
] 

Sreekanth Ramakrishnan commented on MAPREDUCE-1219:
---------------------------------------------------

In todays state, JobTracker publishes its metrics along with its running jobs metrics. The
running jobs list can be pretty long and the metrics updating cycle is done every heartbeat.
This causes a significant increase in heartbeat processing time. Also, the job level metrics
are nothing other than counters of the running job. The counters of running job are obtained
by locking up the job, which also does not help us in terms of performance. But looking at
the information published, shouldn't jobtracker publish its own metrics and not include job
level details? Also, users can obtain the counters using different API. So can we remove the
job level metrics aka counters from JobTracker metrics? Thoughts?

> JobTracker Metrics causes undue load on JobTracker
> --------------------------------------------------
>
>                 Key: MAPREDUCE-1219
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1219
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 0.21.0
>            Reporter: Jothi Padmanabhan
>            Assignee: Amareshwari Sriramadasu
>         Attachments: MAPREDUCE-1219.patch, patch-1219-ydist.txt
>
>
> JobTrackerMetricsInst.doUpdates updates job-level counters of all running jobs into JobTracker's
metrics causing very bad performance and hampers heartbeats. Since Job level metrics are better
served by JobHistory, it may be a good idea to remove these from the metrics framework.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message