Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <1247621270.1224671444317.JavaMail.jira@brutus>
Date: Wed, 22 Oct 2008 03:30:44 -0700 (PDT)
From: "Hemanth Yamijala (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-4472) Should we move out the creation of
 setup/cleanup tasks from JobInProgress.initTasks()?
In-Reply-To: <301694706.1224583426490.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-4472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12641761#action_12641761 ] 

Hemanth Yamijala commented on HADOOP-4472:
------------------------------------------

If we choose to make the schedulers unaware of the setup/cleanup tasks, one more consideration is whether the JT should add the job to the scheduler only after setup task is complete.

Consider this example: two jobs are added J1 and J2. Say J1's setup task takes a long time to complete, and J2's setup task completes very soon. If the scheduler is informed about the two jobs before their setup tasks have completed (as it happens today), the scheduler would look at J1 and initialize its tasks. But since setup has not completed, it would move to initialize the second job as well. Initializing the first job at this point seems to be wastefully occupying JT memory, as no M/R task can run until setup is complete. Put another way, letting task initialization happen only after setup has completed will help to reduce the memory footprint of the JT.

The flip side of this approach is that tasks would be given to the scheduler in order of completion of the setup tasks (irrespective of other aspects like submission time, priority, etc). However, I believe at some point (in 0.19) setup was being done as part of the job client, before they were graduated to tasks. That implicitly meant that jobs are given to the scheduler in order of completion of setup tasks. So, it may not be that bad.

Does this make sense ?

> Should we move out the creation of setup/cleanup tasks from JobInProgress.initTasks()? 
> ---------------------------------------------------------------------------------------
>
>                 Key: HADOOP-4472
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4472
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Vivek Ratan
>
> JobInProgress.initTasks() creates TIPs for map and reduce tasks, and also the newly-introduced setup and cleanup tasks. initTasks() is called by the schedulers, as for reasons of memory optimizations, schedulers may choose to initialize M/R tasks at various moments (the Capacity Scheduler, for example, calls initTasks() just when it considers a job for running). One can say that Schedulers 'own' the initialization of M/R tasks in a job. Furthermore the JT 'owns' the setup and cleanup tasks (it schedules them, and Schedulers are unaware of these tasks). This causes a problematic dependency between the JT and a Scheduler. For example, the Capacity Scheduler calls initTasks() and immediately calls JobInProgress.obtainNewMapTask for a map task. This is a problem today, because we cannot run any map or reduce tasks before the setup task is run, which the Capacity Scheduler is not aware of. 
> Either all Schedulers are explicitly aware of setup/cleanup tasks and their dependencies with M/R tasks (in which case, Schedulers 'own' the creation and scheduling of all these tasks correctly), or the JT 'owns' the setup/cleanup tasks and Schedulers are completely unaware of them (in which case, the creation of setup/cleanup tasks must be moved out of initTasks into a separate method which is called by the JT). 
> I think the latter is the right way to go (unless we implement HADOOP-4421, in which case the former option may be viable as well). 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.