hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vivek Ratan (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-4413) Capacity Scheduler to provide a scheduler history log to record actions taken and why
Date Wed, 14 Jan 2009 08:52:59 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-4413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Vivek Ratan updated HADOOP-4413:

    Attachment: 4413.1.patch

Attaching the first patch (4413.1.patch), to get feedback on the approach.

* Much like _JobTrackerInstrumentation_, I've defined a _CapacitySchedulerInstrumentation_
class that defines the events and data we want to capture for the Capacity Scheduler. 
* There is a single 'implementation' of this class, _CapacitySchedulerLogInst_, that writes
stuff to a log file. We may have implementations in the future that interact with Chukwa directly.
* The time series data is captured the same way the scheduler UI does. The Capacity Scheduler
provides an object whose toString() method generates all the data that needs to be captured.
This is the same object used by the UI. A thread in _CapacitySchedulerLogInst_ periodically
writes this data to a log file. The default period is 5 seconds, but can be overwritten through
the capacity scheduler's configuration. 
* Events are written to the log file right away. If this proves to be expensive, we can buffer
them up (in a simple linked list of strings, perhaps) and write them periodically as well.
These events capture the main scheduler decisions. 

To be done: 
* update documentation on capacity scheduler configuration
* log4J settings to log to a separate file
* make sure we're capturing all relevant events

Feedback welcome.

> Capacity Scheduler to provide a scheduler history log to record actions taken and why
> -------------------------------------------------------------------------------------
>                 Key: HADOOP-4413
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4413
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: contrib/capacity-sched
>            Reporter: Mac Yang
>         Attachments: 4413.1.patch
> It would be very useful if the capacity scheduler can provide a log that record the decisions
made and actions taken by the scheduler.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message