Mailing-List: contact mapreduce-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-dev@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of sandy.ryza@cloudera.com
 designates 209.85.216.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CA+01ahhQ_-hDTeCBdZh6z-F4RrvZaofExho-33UGoAUgSiUg+A@mail.gmail.com>
References: 
 <CA+01ahhQ_-hDTeCBdZh6z-F4RrvZaofExho-33UGoAUgSiUg+A@mail.gmail.com>
Date: Thu, 18 Oct 2012 16:58:29 -0700
Message-ID: 
 <CACBYxKLaH_SNBxN=G+YNH9+DrKf6BVsznFiKb7jEpQXA4Mw9YQ@mail.gmail.com>
Subject: Re: Max Maps for default queues in FairScheduler
From: Sandy Ryza <sandy.ryza@cloudera.com>
To: mapreduce-dev@hadoop.apache.org
Content-Type: multipart/alternative; boundary=047d7b6da4ccf074b004cc5e29a5

--047d7b6da4ccf074b004cc5e29a5
Content-Type: text/plain; charset=ISO-8859-1

You're correct that there's no way to put a hard limit on the number of
maps or reduces for a given user, and a user can potentially consume all of
the cluster resources.  However, if there are multiple users contending for
resources, the scheduler makes an effort to schedule tasks equally, so it
would be unlikely for a single user to get that much of the cluster.

Can I ask what you need a userMaxMaps/Reducers for?

On Thu, Oct 18, 2012 at 4:41 PM, lohit <lohit.vijayarenu@gmail.com> wrote:

> I am trying to understand FairScheduler configs I am trying to see if there
> is a way to achieve the below.
> I see that if there are no pools configured (or only few pools are
> configured ) and a user submits a job, it would end up in his own pool,
> right?
> Now, I see there are some limits you can set globally for such users, for
> example userMaxJobsDefault.
> Is there a way to set userMaxMaps or userMaxReducers? It looks like if I
> have few pools configured and a user who submits a job without specify a
> pool will be given his own pool. He can potentially consume 100% of
> Map/Reduce slots. Is my understand correct?
>
> --
> Have a Nice Day!
> Lohit
>

--047d7b6da4ccf074b004cc5e29a5--