hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3930) Decide how to integrate scheduler info into CLI and job tracker web page
Date Wed, 27 Aug 2008 09:25:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12626042#action_12626042
] 

Hemanth Yamijala commented on HADOOP-3930:
------------------------------------------

I had an offline conversation with Owen and we came to the following proposal:

- While the usability of the UI is enhanced by displaying a one-column per queue / job scheduling
attribute, in the interest of simplicity, we are proposing to display the information as a
single string in a single column.
- This information would be available via a {{toString()}} API on the SchedulerInfo object
proposed by Owen above.
-- One of the most important reasons to do it this way is to keep in mind that this information
needs to be consumed by the CLI too, which should be transferred on wire. Passing something
like a map is going to be tricky for the framework.
-- Also, as seen from discussions above, requiring additional APIs to determine the column
order etc become unnecessary if we assume the scheduler will take care of formatting the string
in the scheduler info as it pleases. This makes the API simpler.
- Regarding getting the scheduler info per queue, Owen proposed adding this to the {{QueueManager}}
class being discussed in HADOOP-3698. Something like:
{code}
class QueueManager {
  public void setSchedulingInfo(String queue, Object queueInfo);
  public Object getSchedulingInfo(String queue);
}
{code}



> Decide how to integrate scheduler info into CLI and job tracker web page
> ------------------------------------------------------------------------
>
>                 Key: HADOOP-3930
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3930
>             Project: Hadoop Core
>          Issue Type: Improvement
>    Affects Versions: 0.17.2
>            Reporter: Matei Zaharia
>            Assignee: Sreekanth Ramakrishnan
>            Priority: Minor
>         Attachments: 3930-1.patch, mockup.JPG
>
>
> We need a way for job schedulers such as HADOOP-3445 and HADOOP-3476 to provide info
to display on the JobTracker web interface and in the CLI. The main things needed seem to
be:
> * A way for schedulers to provide info to show in a column on the web UI and in the CLI
- something as simple as a single string, or a map<string, int> for multiple parameters.
> * Some sorting order for jobs - maybe a method to sort a list of jobs.
> Let's figure out what the best way to do this is and implement it in the existing schedulers.
> My first-order proposal at an API: Augment the TaskScheduler with
> * public Map<String, String> getSchedulingInfo(JobInProgress job) -- returns key-value
pairs which are displayed in columns on the web UI or the CLI for the list of jobs.
> * public Map<String, String> getSchedulingInfo(String queue) -- returns key-value
pairs which are displayed in columns on the web UI or the CLI for the list of queues.
> * public Collection<JobInProgress> getJobs(String queueName) -- returns the list
of jobs in a given queue, sorted by a scheduler-specific order (the order it wants to run
them in / schedule the next task in / etc).
> * public List<String> getQueues();

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message