hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4576) Modify pending tasks count in the UI to pending jobs count in the UI
Date Thu, 04 Dec 2008 06:58:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653209#action_12653209

Amar Kamat commented on HADOOP-4576:

Comments :
{{JobQueuesManager.java}} :
1) No need to import {{AtomicInteger}}. Also plz remove extra diffs.

{{CapacityTaskScheduler.java}} :
1) You cant simply change the constructor def. Overload it and deprecate the other if needed.

2) Extra diffs w.r.t. 
-         supportsPriority?"YES":"NO"));      
+         supportsPriority?"YES":"NO"));

3) {quote}
+      sb.append(String.format("* Scheduling information can be off by " +
+      		"maximum of %d seconds\n", pollingInterval/1000));
Use {{StringUtils}} for formatting time.

1) {quote}
+    scheduler.assignTasks(tracker("tt1")); // heartbeat
+    p.selectJobsToInitialize();
The ordering doesnt seem right. You submit 5 jobs, try to assign tasks (which should be no-op)
and then you init the jobs.

2) Shouldnt we also check/test the timing issue that after _poll-interval_ units of time the
values are correct. Something like 
  - add jobs
  - check the queue-sched-info
  - allow the jobs to be inited by the poller i.e wait for _poller-interval_ time
  - check again to see if the change is made.

2) Plz mark the start and end of a new sub-test using comments

> Modify pending tasks count in the UI to pending jobs count in the UI
> --------------------------------------------------------------------
>                 Key: HADOOP-4576
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4576
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: contrib/capacity-sched
>    Affects Versions: 0.19.0
>            Reporter: Hemanth Yamijala
>            Assignee: Sreekanth Ramakrishnan
>         Attachments: HADOOP-4576-1.patch, HADOOP-4576-2.patch, HADOOP-4576-3.patch
> The UI for capacity scheduler displays 'pending tasks' counts. However the capacity scheduler
does not update these counts to be the actual values for optimization purposes, for e.g. to
avoid walking all pending jobs on all heartbeats. Hence this information is not very accurate.
> Also, while 'running tasks' counts are useful to compare against capacities and limits,
'pending tasks' counts do not add too much user value. A better count to display would be
the number of running and pending jobs.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message