hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ari Rabkin (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-657) Free temporary space should be modelled better
Date Wed, 21 May 2008 22:27:55 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12598828#action_12598828
] 

Ari Rabkin commented on HADOOP-657:
-----------------------------------

Here's my proposed fix:

1) Add a "free space on compute node" field to TaskTrackerStatus.	This is the real physical
space available, plus the sum of (commitment - reservation) for each running map task.

2) Add a "space used by this task" and "space reserved for task" to TaskStatus as well.  

3) Add a "space to reserve" to either Task or MapTask.  This is computed by the JobTracker,
and used by the TaskTracker

4) Create a new ResourceConsumptionEstimator class, and have an instance of that type for
each JobInProgress.   This will have, at a minimum, reportCompletedMapTask(MapTaskStatus t)
and estimateSpaceForMapTask(MapTask mt)     The implementation would probably be a thread
that processes asynchronously, and updates an atomic value that'll be either the estimated
space requirement, or else the estimated ratio between input size and output size.   Until
sufficiently many maps have completed (10%, say) the size estimate would just be the size
of each map's input.  Afterwards, we'll take the 75th percentile of the measured blowup in
task size.

5) Modify obtainNewMapTask to return null if the space available on the given task tracker
is less than the estimate of available space.  

6) To avoid deadlocks if there are multiple jobs contending for space, abort the job if too
many trackers are rejected as having insufficient space.  

-----
Thoughts?

> Free temporary space should be modelled better
> ----------------------------------------------
>
>                 Key: HADOOP-657
>                 URL: https://issues.apache.org/jira/browse/HADOOP-657
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.7.2
>            Reporter: Owen O'Malley
>            Assignee: Ari Rabkin
>
> Currently, there is a configurable size that must be free for a task tracker to accept
a new task. However, that isn't a very good model of what the task is likely to take. I'd
like to propose:
> Map tasks:  totalInputSize * conf.getFloat("map.output.growth.factor", 1.0) / numMaps
> Reduce tasks: totalInputSize * 2 * conf.getFloat("map.output.growth.factor", 1.0) / numReduces
> where totalInputSize is the size of all the maps inputs for the given job.
> To start a new task, 
>   newTaskAllocation + (sum over running tasks of (1.0 - done) * allocation) >= 
>        free disk * conf.getFloat("mapred.max.scratch.allocation", 0.90);
> So in English, we will model the expected sizes of tasks and only task tasks that should
leave us a 10% margin. With:
> map.output.growth.factor -- the relative size of the transient data relative to the map
inputs
> mapred.max.scratch.allocation -- the maximum amount of our disk we want to allocate to
tasks.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message