Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAAEYuvf39HyqZ3OmR=w_gOsb3uDHQ0mrAfwG0BTYZUqOWB15ig@mail.gmail.com>
References: 
 <CAAEYuvf8U3sLvHviYH_yspL==NSGODACztfOgXwCWdDuoBDPdQ@mail.gmail.com>
	<CAAEYuvf39HyqZ3OmR=w_gOsb3uDHQ0mrAfwG0BTYZUqOWB15ig@mail.gmail.com>
Date: Thu, 17 Sep 2015 16:17:43 +0530
Message-ID: 
 <CAAntRUJfBQhmUwar6o+qROvkRxDYfcD4Ky2odV2Qc7CsgDUXDw@mail.gmail.com>
Subject: Re: Concurrency control
From: Naganarasimha Garla <naganarasimha.gr@gmail.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=001a113dc334127406051fef2522

--001a113dc334127406051fef2522
Content-Type: text/plain; charset=UTF-8

Hi Laxman,
Yes if cgroups are enabled and
"yarn.scheduler.capacity.resource-calculator" configured
to DominantResourceCalculator then cpu and memory can be controlled.
Please Kindly  furhter refer to the official documentation
http://hadoop.apache.org/docs/r1.2.1/capacity_scheduler.html

But may be if say more about problem then we can suggest ideal
configuration, seems like capacity configuration and splitting of the queue
is not rightly done or you might refer to Fair Scheduler if you want more
fairness for container allocation for different apps.

On Thu, Sep 17, 2015 at 4:10 PM, Laxman Ch <laxman.lux@gmail.com> wrote:

> Hi,
>
> In YARN, do we have any way to control the amount of resources (vcores,
> memory) used by an application SIMULTANEOUSLY.
>
> - In my cluster, noticed some large and long running mr-app occupied all
> the slots of the queue and blocking other apps to get started.
> - I'm using Capacity schedulers (using hierarchical queues and preemption
> disabled)
> - Using Hadoop version 2.6.0
> - Did some googling around this and gone through configuration docs but
> I'm not able to find anything that matches my requirement.
>
> If needed, I can provide more details on the usecase and problem.
>
> --
> Thanks,
> Laxman
>

--001a113dc334127406051fef2522
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Laxman,<div>Yes if cgroups are enabled and &quot;<span =
style=3D"color:rgb(51,51,51);font-family:monospace;font-size:11px;backgroun=
d-color:rgb(223,227,219)">yarn.scheduler.capacity.resource-calculator&quot;=
</span>=C2=A0configured to=C2=A0<span style=3D"color:rgb(51,51,51);font-fam=
ily:Verdana,Helvetica,Arial,sans-serif;font-size:11px;background-color:rgb(=
223,227,219)">DominantResourceCalculator=C2=A0</span>then cpu and memory ca=
n be controlled.</div><div><span style=3D"font-size:12.8px">Please Kindly =
=C2=A0furhter refer to the official documentation=C2=A0</span><br style=3D"=
font-size:12.8px"><a href=3D"http://hadoop.apache.org/docs/r1.2.1/capacity_=
scheduler.html" style=3D"font-size:12.8px" target=3D"_blank">http://hadoop.=
apache.org/docs/r1.2.1/capacity_scheduler.html</a><br></div><div><br></div>=
<div>But may be if say more about problem then we can suggest ideal configu=
ration, seems like capacity configuration and splitting of the queue is not=
 rightly done or you might refer to Fair Scheduler if you want more fairnes=
s for container allocation for different apps.</div></div><div class=3D"gma=
il_extra"><br><div class=3D"gmail_quote">On Thu, Sep 17, 2015 at 4:10 PM, L=
axman Ch <span dir=3D"ltr">&lt;<a href=3D"mailto:laxman.lux@gmail.com" targ=
et=3D"_blank">laxman.lux@gmail.com</a>&gt;</span> wrote:<br><blockquote cla=
ss=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;pa=
dding-left:1ex"><div class=3D"HOEnZb"><div class=3D"h5"><div dir=3D"ltr"><d=
iv><div class=3D"gmail_quote"><div dir=3D"ltr">Hi,<div><br></div><div>In YA=
RN, do we have any way to control the amount of resources (vcores, memory) =
used by an application SIMULTANEOUSLY.</div><div><br></div><div><div>- In m=
y cluster, noticed some large and long running mr-app occupied all the slot=
s of the queue and blocking other apps to get started.</div><div>- I&#39;m =
using Capacity schedulers (using hierarchical queues and preemption disable=
d)=C2=A0</div><div>- Using Hadoop version 2.6.0</div><div>- Did some googli=
ng around this and gone through configuration docs but I&#39;m not able to =
find anything that matches my requirement.</div><div><br></div><div>If need=
ed, I can provide more details on the usecase and problem.</div></div></div=
></div><div><br></div>-- <br><div>Thanks,<br>Laxman</div>
</div></div>
</div></div></blockquote></div><br></div>

--001a113dc334127406051fef2522--