hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4413) Capacity Scheduler to provide a scheduler history log to record actions taken and why
Date Thu, 22 Jan 2009 18:35:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12666245#action_12666245

Hemanth Yamijala commented on HADOOP-4413:

Vivek, if we want to correlate events about a job from two disparate logs - the jt logs and
the capacity scheduler logs, there must be some key that should tie them together, right ?
I thought that should be the job id or in case of tasks, the task id. These fields should
be there in both the logs. Am I missing something here ?

bq. Plus, we don't want too many changes to CapacitySchedulerInstrumentation - it acts like
an interface.

Agreed. +1. So, let's leave the scheduler instance in.

bq. Again, I sense that what all we want to capture will become clearer once we run this thing
and start analyzing life cycle events. I've tried to capture whatever I thought would be important.
But feel free to suggest other events.

I can see this is going to be an ongoing effort. So, your argument about adding new events
as the need arises seems very valid. So, let's ignore thinking about new events. 

For the events defined now, the general approach I would take is to include more information
than less - so it leaves options open. Again for same reasons, because this is an interface,
we may not want to frequently change it as we see the need to do so. So, I think there are
only 2 I can add:

foundMapTask: Include Jobid and taskid
blockonHighMemJob: include jobid

Rest look fine.

> Capacity Scheduler to provide a scheduler history log to record actions taken and why
> -------------------------------------------------------------------------------------
>                 Key: HADOOP-4413
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4413
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: contrib/capacity-sched
>            Reporter: Mac Yang
>         Attachments: 4413.1.patch
> It would be very useful if the capacity scheduler can provide a log that record the decisions
made and actions taken by the scheduler.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message