Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of harsh@cloudera.com designates
 209.85.223.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAMWNowECAVNS=2zqotDUuO8AZROVeGg248QJcvyFNxHnwULDdA@mail.gmail.com>
References: <6203AF4C-72E6-43C7-B712-0CF2CD63BA81@gmail.com>
 <CAOcnVr2SxrCUd5xLjuKa-Qp_Fh7hticrEASLJQ4bUZ8L-=XAMA@mail.gmail.com>
 <CAMWNowECAVNS=2zqotDUuO8AZROVeGg248QJcvyFNxHnwULDdA@mail.gmail.com>
From: Harsh J <harsh@cloudera.com>
Date: Tue, 16 Oct 2012 03:48:14 +0530
Message-ID: 
 <CAOcnVr1RE7ZQNPrX8CoLgwpMv8-wJFo71ppHqMtdVUR4b+udTA@mail.gmail.com>
Subject: Re: Fair scheduler.
To: user@hadoop.apache.org
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

Hi Patai,

Reply inline.

On Tue, Oct 16, 2012 at 2:57 AM, Patai Sangbutsarakum
<silvianhadoop@gmail.com> wrote:
> Thanks for input,
>
> I am reading the document; i forget to mention that i am on cdh3u4.

That version should have the support for all of this.

>> If you point your poolname property to mapred.job.queue.name, then you
>> can leverage the Per-Queue ACLs
>
> Is that mean if i plan to 3 pools of fair scheduler, i have to
> configure 3 queues of capacity scheduler. in order to have each pool
> can leverage Per-Queue ACL of each queue.?

Queues are not hard-tied into CapacityScheduler. You can have generic
queues in MR. And FairScheduler can bind its Pool concept into the
Queue configuration.

All you need to do is the following:

1. Map FairScheduler pool name to reuse queue names itself:

mapred.fairscheduler.poolnameproperty set to 'mapred.job.queue.name'

2. Define your required queues:

mapred.job.queues set to "default,foo,bar" for example, for 3 queues:
default, foo and bar.

3. Define Submit ACLs for each Queue:

mapred.queue.default.acl-submit-job set to "patai,foobar users,adm"
(usernames groupnames)

mapred.queue.foo.acl-submit-job set to "spam eggs"

Likewise for remaining queues, as you need it=85

4. Enable ACLs and restart JT.

mapred.acls.enabled set to "true"

5. Users then use the right API to set queue names before submitting
jobs, or use -Dmapred.job.queue.name=3Dvalue via CLI (if using Tool):
http://hadoop.apache.org/docs/stable/api/org/apache/hadoop/mapred/JobConf.h=
tml#setQueueName(java.lang.String)

6. Done.

Let us know if this works!

--=20
Harsh J