Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Date: Sat, 27 Aug 2011 00:07:30 +0000 (UTC)
From: "Ahmed Radwan (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: 
 <959368361.20073.1314403650602.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1501748263.30519.1313097987253.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Updated] (MAPREDUCE-2836) Provide option to fail jobs when
 submitted to non-existent pools.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/MAPREDUCE-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ahmed Radwan updated MAPREDUCE-2836:
------------------------------------

    Attachment: MAPREDUCE-2836_rev2.patch

Many thanks Matei for your comments! 

I have updated the patch.

- I agree, the "mapred.fairscheduler.allowUndeclaredPools" is a more descriptive property name, I changed it.

- When a user tries to add a job to an undeclared pool, the exception thrown by PoolManager.addJob(..) will go through FairScheduler.JobListner.jobAdded(..), JobTracker.addJob(..), JobTracker.submitJob(..), JobClient.submitJobInternal(..), and finally thrown by Job.submit(..). This shouldn't cause the JobTracker to crash, but is a noisy way of communicating the problem. 

- I have also created a new exception UndeclaredPoolException instead of the IllegalArgumentException.


> Provide option to fail jobs when submitted to non-existent pools.
> -----------------------------------------------------------------
>
>                 Key: MAPREDUCE-2836
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2836
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: contrib/fair-share
>            Reporter: Jeff Bean
>            Assignee: Ahmed Radwan
>            Priority: Minor
>         Attachments: MAPREDUCE-2836.patch, MAPREDUCE-2836_rev2.patch
>
>
> In some environments, it might be desirable to explicitly specify the fair scheduler pools and to explicitly fail jobs that are not submitted to any of the pools. 
> Current behavior of the fair scheduler is to submit jobs to a default pool if a pool name isn't specified or to create a pool with the new name if the pool name doesn't already exist. There should be a configuration option for the fair scheduler that causes it to noisily fail the job if it's submitted to a pool that isn't pre-specified or if the specified pool doesn't exist.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira