hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "rahul k singh (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5930) Instrument events in the capacity scheduler for collecting metrics information
Date Tue, 16 Jun 2009 09:24:07 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12720006#action_12720006
] 

rahul k singh commented on HADOOP-5930:
---------------------------------------

Following metrics would be collected.

This would help in finding out no of heartbeat wasted.
  1. assignTasks returns null 

How many times mapscheduler is invoked vs how many times reduceScheduler is invoked
  2 .mapscheduler is invoked,
  3 .reduceScheduler is invoked,
  4 .Task scheduled from queue
  5 .Task scheduled from queue ignoring user limits
  6 High RAM job's task scheduled from queue

Calculating skip count incase of following.
  7 Task skipped due to user limit exceeding with reason
  8 Task skipped due to high RAM jobs with reason

  9 Priority of job changed
 10 #of times initializer skips initializing pending jobs.

status events.
 11 Job becomes running
 12 Job Added
 13 failed jobs in queue
 14 killed jobs in queue
 15 completed jobs in queue

Queue statistics.
 16 running tasks in queue and per user
 17 pending tasks in queue
 18 failed/kill tasks in queue
 19 Amount of time queue is over capacity

> Instrument events in the capacity scheduler for collecting metrics information
> ------------------------------------------------------------------------------
>
>                 Key: HADOOP-5930
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5930
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: contrib/capacity-sched
>            Reporter: Hemanth Yamijala
>
> We need to instrument various events in the capacity scheduler so that we can collect
metrics about them. This data will help us determine improvements to scheduling strategies
itself.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message