Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of ojoshi@hortonworks.com
 designates 209.85.215.49 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAKtYt7AQSmCQ5_3DYFPzCkWJocgRdzFYV8BrjO6rQKGP1tG1vQ@mail.gmail.com>
References: 
 <CAKtYt7AQSmCQ5_3DYFPzCkWJocgRdzFYV8BrjO6rQKGP1tG1vQ@mail.gmail.com>
Date: Tue, 19 Nov 2013 14:01:32 -0800
Message-ID: 
 <CABcwWrgyLLjKQAiO0Cd1bRsW3Kj+bzx2ryEMWY9XrVVrt6Z25A@mail.gmail.com>
Subject: Re: Limit on total jobs running using fair scheduler
From: Omkar Joshi <ojoshi@hortonworks.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=001a11c2672ab94db704eb8eced1

--001a11c2672ab94db704eb8eced1
Content-Type: text/plain; charset=US-ASCII

Not sure about the fair scheduler but in capacity scheduler you can achieve
this by controlling number of jobs/applications per queue.

Thanks,
Omkar Joshi
*Hortonworks Inc.* <http://www.hortonworks.com>


On Tue, Nov 19, 2013 at 3:26 AM, Ivan Tretyakov <itretyakov@griddynamics.com
> wrote:

> Hello!
>
> We are using CDH 4.1.1 (Version: 2.0.0-mr1-cdh4.1.1) and fair-scheduler.
> We need to limit total number of jobs which can run at the same time on
> cluster.
> I can see maxRunningJobs options but it sets limit for pool or user.
> We wouldn't like to limit each pool or user we just need to set limit on
> total number of jobs running.
>
> Is it possible to do it using fair scheduler?
> Can capacity scheduler help here?
> Maybe there are other options to achieve the goal?
>
> Thanks in advance!
>
> --
> Best Regards
> Ivan Tretyakov
>
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

--001a11c2672ab94db704eb8eced1
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Not sure about the fair scheduler but in capacity schedule=
r you can achieve this by controlling number of jobs/applications per queue=
.<div class=3D"gmail_extra"><br clear=3D"all"><div><div dir=3D"ltr"><font f=
ace=3D"courier new, monospace">Thanks,</font><div>
<font face=3D"courier new, monospace">Omkar Joshi</font></div><div><font fa=
ce=3D"courier new, monospace"><a href=3D"http://www.hortonworks.com" target=
=3D"_blank"><b>Hortonworks Inc.</b></a></font></div></div></div>
<br><br><div class=3D"gmail_quote">On Tue, Nov 19, 2013 at 3:26 AM, Ivan Tr=
etyakov <span dir=3D"ltr">&lt;<a href=3D"mailto:itretyakov@griddynamics.com=
" target=3D"_blank">itretyakov@griddynamics.com</a>&gt;</span> wrote:<br><b=
lockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px =
#ccc solid;padding-left:1ex">
<div dir=3D"ltr">Hello!<div><br></div><div>We are using CDH 4.1.1 (Version:=
 2.0.0-mr1-cdh4.1.1) and fair-scheduler.</div><div>We need to limit total n=
umber of jobs which can run at the same time on cluster.</div><div>I can se=
e=A0maxRunningJobs options but it sets limit for pool or user.</div>


<div>We wouldn&#39;t like to limit each pool or user we just need to set li=
mit on total number of jobs running.</div><div><br></div><div>Is it possibl=
e to do it using fair scheduler?</div><div>Can capacity scheduler help here=
?=A0</div>


<div>Maybe there are other options to achieve the goal?</div><div><br></div=
><div>Thanks in advance!</div><span class=3D"HOEnZb"><font color=3D"#888888=
"><div><div><br></div>-- <br><div dir=3D"ltr"><div>Best Regards</div><div>I=
van Tretyakov</div>
<div><br></div>
</div>
</div></font></span></div>
</blockquote></div><br></div></div>

<br>
<span style=3D"color:rgb(128,128,128);font-family:Arial,sans-serif;font-siz=
e:10px">CONFIDENTIALITY NOTICE</span><br style=3D"color:rgb(128,128,128);fo=
nt-family:Arial,sans-serif;font-size:10px"><span style=3D"color:rgb(128,128=
,128);font-family:Arial,sans-serif;font-size:10px">NOTICE: This message is =
intended for the use of the individual or entity to which it is addressed a=
nd may contain information that is confidential, privileged and exempt from=
 disclosure under applicable law. If the reader of this message is not the =
intended recipient, you are hereby notified that any printing, copying, dis=
semination, distribution, disclosure or forwarding of this communication is=
 strictly prohibited. If you have received this communication in error, ple=
ase contact the sender immediately and delete it from your system. Thank Yo=
u.</span>
--001a11c2672ab94db704eb8eced1--