Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Date: Thu, 1 Sep 2011 18:03:09 +0000 (UTC)
From: "Todd Lipcon (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: 
 <1413228124.7668.1314900189994.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1501748263.30519.1313097987253.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (MAPREDUCE-2836) Provide option to fail jobs
 when submitted to non-existent pools.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/MAPREDUCE-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13095478#comment-13095478 ] 

Todd Lipcon commented on MAPREDUCE-2836:
----------------------------------------

A few comments on the patch:
- the new exception type needs a serialVersionUID
- Rather than call conf.getBoolean() every time you need to look at this configuration, assign a member field in PoolManager (like is done for allocFile, etc)
- I find the two different getPool() methods a little confusing. Perhaps you could do have the following methods:
-- public synchronized Pool getPool(String name): return the specified pool, throwing if it doesn't exist (the behavior when your 'create' param is false)
-- private synchronized Pool getOrCreatePool(String name): return the pool if it exists. If it doesn't exist, and the conf is true, create it and return it. If it doesn't exist and the conf is false, throw UndeclaredPoolException.
- small nit: in the exception message for UndeclaredPoolException, put quotes around the pool name: "Pool name: '" + name + "' ..." -- this makes it easier to debug if the user accidentally has some trailing whitespace causing a problem. You might consider having this message include something like "Valid pools are: " + StringUtils.join(", ", pools.keySet())

Does that make sense?

> Provide option to fail jobs when submitted to non-existent pools.
> -----------------------------------------------------------------
>
>                 Key: MAPREDUCE-2836
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2836
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: contrib/fair-share
>            Reporter: Jeff Bean
>            Assignee: Ahmed Radwan
>            Priority: Minor
>         Attachments: MAPREDUCE-2836.patch, MAPREDUCE-2836_rev2.patch, MAPREDUCE-2836_rev3.patch
>
>
> In some environments, it might be desirable to explicitly specify the fair scheduler pools and to explicitly fail jobs that are not submitted to any of the pools. 
> Current behavior of the fair scheduler is to submit jobs to a default pool if a pool name isn't specified or to create a pool with the new name if the pool name doesn't already exist. There should be a configuration option for the fair scheduler that causes it to noisily fail the job if it's submitted to a pool that isn't pre-specified or if the specified pool doesn't exist.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira