hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4018) limit memory usage in jobtracker
Date Mon, 06 Oct 2008 06:09:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12637007#action_12637007
] 

Amar Kamat commented on HADOOP-4018:
------------------------------------

Few comments
1) 
{code}
int maxTasks = conf.getInt("mapred.jobtracker.maxtasks.per.job", -1);
{code}
This will change per job as _conf_ is configured per job. Hence all the jobs should get this
value from the jobtracker. Something like
{code}
-----JobTracker-------------
int getMaxTasksPerJob() {
  return conf.get("mapred.jobtracker.maxtasks.per.job", -1);
}
-----JobInProgress---------
int maxTasks = jobtracker.getMaxTasks();
{code}
2) 
{code}
if (maxTasks != -1 && numMapTasks + numReduceTasks > maxTasks) {
{code}
What is I pass _-2_? So the check should be 
{code}
if (maxTasks > 0 && numMapTasks + numReduceTasks > maxTasks) {
{code}
3) 
{code}
if (maxTasks != -1 && numMapTasks + numReduceTasks > maxTasks) {
      long now = System.currentTimeMillis();
      this.finishTime = now;
      status.setRunState(JobStatus.FAILED);
      JobHistory.JobInfo.logFailed(profile.getJobID(), now, 0, 0);
      // Special case because the Job is not queued
      JobEndNotifier.registerNotification(this.getJobConf(), this.getStatus());
      throw new IOException(
                "The number of tasks for this job " + 
                (numMapTasks + numReduceTasks) +
                " exceeds the configured limit " + maxTasks);
}
{code}
can be written as 
{code}
if (maxTasks > 0 && numMapTasks + numReduceTasks > maxTasks) {
      throw new IOException(
                "The number of tasks for this job " + 
                (numMapTasks + numReduceTasks) +
                " exceeds the configured limit " + maxTasks);
 }
{code}
Note that whatever you have done should be done by the job upon failure i.e
  - setting the runstate to {{FAILED}}
  - setting the finish time
  - logging failure to history and hence closing the file.

4) One minor nit. _mapred.jobtracker.maxtasks.per.job_ could be _mapred.job.tasks.maximum_

The test case looks fine. Plz check with everyone if passing -1 for _UNLIMITED_ is fine. Wondering
what is 0 is passed? Should be bail out immediately or should we consider any non-positive
number as _UNLIMITED_?

> limit memory usage in jobtracker
> --------------------------------
>
>                 Key: HADOOP-4018
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4018
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>         Attachments: maxSplits.patch, maxSplits2.patch, maxSplits3.patch, maxSplits4.patch,
maxSplits5.patch, maxSplits6.patch, maxSplits7.patch, maxSplits8.patch, maxSplits9.patch
>
>
> We have seen instances when a user submitted a job with many thousands of mappers. The
JobTracker was running with 3GB heap, but it was still not enough to prevent memory trashing
from Garbage collection; effectively the Job Tracker was not able to serve jobs and had to
be restarted.
> One simple proposal would be to limit the maximum number of tasks per job. This can be
a configurable parameter. Is there other things that eat huge globs of memory in job Tracker?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message