Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 97FC6EE02 for ; Wed, 16 Jan 2013 18:08:22 +0000 (UTC) Received: (qmail 7635 invoked by uid 500); 16 Jan 2013 18:08:17 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 7523 invoked by uid 500); 16 Jan 2013 18:08:17 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 7516 invoked by uid 99); 16 Jan 2013 18:08:17 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 16 Jan 2013 18:08:17 +0000 X-ASF-Spam-Status: No, hits=1.8 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of bugcy013@gmail.com designates 209.85.212.48 as permitted sender) Received: from [209.85.212.48] (HELO mail-vb0-f48.google.com) (209.85.212.48) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 16 Jan 2013 18:08:12 +0000 Received: by mail-vb0-f48.google.com with SMTP id fc21so1617449vbb.21 for ; Wed, 16 Jan 2013 10:07:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:content-type; bh=NCVtnRRU9A2wUzGfnOyDidbmtHPI4O1gIprTKr19n24=; b=bBy6cPliXuRu77ZHT7ucpf9mcS1G8rXjsfAC0fQOQ0pDRn3UZyqmkAcyzmNJiLIC9G HglpHSXs3VtfsrhMLeH/UskuBMkvuMo0BVfKcLZa86F/a16bICqO5hdf3wgCh5aYMvNC YIUrpZQB7cxAsP9Lugsn0/MQsv6Yp9/Pyt6YNh5pUM2Cq3x9/9aoe83jthsUeQNdGmlZ s1JHcqtUb3o9bIAg50nn/6INjyZ5c1ukhzD8z+nfXqQsE50bK0agj2qRXMsowb0Hb26g af1NEXMpdVbDp0z7Z4724gvdidpVE7OKmSfhLpR76+WCYeQukgFXQmOA2QfZdgnCdqyM pPPA== X-Received: by 10.52.66.71 with SMTP id d7mr1929844vdt.121.1358359671574; Wed, 16 Jan 2013 10:07:51 -0800 (PST) MIME-Version: 1.0 Received: by 10.58.46.161 with HTTP; Wed, 16 Jan 2013 10:07:30 -0800 (PST) In-Reply-To: References: <1899023892924472935@unknownmsgid> <8F12D028AC0A440081FAC86A2521A8F0@gmail.com> <9586818781574C62A4004A5FA69F118D@gmail.com> From: Dhanasekaran Anbalagan Date: Wed, 16 Jan 2013 13:07:30 -0500 Message-ID: Subject: Re: Fair Scheduler is not Fair why? To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=20cf30781366b3924004d36bc144 X-Virus-Checked: Checked by ClamAV on apache.org --20cf30781366b3924004d36bc144 Content-Type: text/plain; charset=UTF-8 HI Jeff, thanks for kindly mail, I have tested sleep job working pretty good. But we have tested with Hadoop streaming job not proper with fair scheduling Algorithm why?. Any other way test Hadoop streaming job, with fair scheduler Note: Tested with RHadoop with rmr. -Dhanasekaran. Did I learn something today? If not, I wasted it. On Wed, Jan 16, 2013 at 12:02 PM, Jeff Bean wrote: > Validate your scheduler capacity and behavior by using sleep jobs. Submit > sleep jobs to the pools that mirror your production jobs and just check > that the scheduler pool allocation behaves as you expect. The nice thing > about sleep is that you can mimic your real jobs: numbers of tasks and how > long they run. > > You should be able to determine that the hypothesis posed on this thread > is correct: that all the slots are taken by other tasks. Indeed, your UI > says that research has 90 running tasks after having completed over 4000, > but your emails says no tasks are scheduled. I'm a little confused. > > Jeff > > > On Wed, Jan 16, 2013 at 8:50 AM, Nan Zhu wrote: > >> BTW, what I mentioned is fairsharepreemption not minimum share >> >> an alternative way to achieve that is to set minimum share of two queues >> to be equal(or other allocation scheme you like), and sum of them is equal >> to the capacity of the cluster, and enable minimumSharePreemption >> >> Good Luck! >> >> Best, >> >> -- >> Nan Zhu >> School of Computer Science, >> McGill University >> >> >> On Wednesday, 16 January, 2013 at 11:43 AM, Nan Zhu wrote: >> >> I think you should do that, so that when the allocation is inconsistent >> with fair share, the tasks in the queue which occupies more beyond it's >> fair share will be killed, and the available slots would be assigned to the >> other one (assuming the weights of them are the same) >> >> Best, >> >> -- >> Nan Zhu >> School of Computer Science, >> McGill University >> >> >> On Wednesday, 16 January, 2013 at 11:32 AM, Dhanasekaran Anbalagan wrote: >> >> HI Nan, >> >> We have not enabled Fair Scheduler Preemption. >> >> -Dhanasekaran. >> >> Did I learn something today? If not, I wasted it. >> >> >> On Wed, Jan 16, 2013 at 11:21 AM, Nan Zhu wrote: >> >> have you enabled task preemption? >> >> Best, >> >> -- >> Nan Zhu >> School of Computer Science, >> McGill University >> >> >> On Wednesday, 16 January, 2013 at 10:45 AM, Justin Workman wrote: >> >> Looks like weight for both pools is equal and all map slots are used. >> Therefore I don't believe anyone has priority for the next slots. Try >> setting research weight to 2. This should allow research to take slots as >> tech released them. >> >> Sent from my iPhone >> >> On Jan 16, 2013, at 8:26 AM, Dhanasekaran Anbalagan >> wrote: >> >> HI Guys >> >> We configured fair scheduler with cdh4, Fair scheduler not work properly. >> Map Task Capacity = 1380 >> Reduce Task Capacity = 720 >> >> We create two users tech and research, we configured equal weight 1 But, >> I stared job in research user mapper will not allocated why? >> please guide me guys. >> >> >> >> >> 5 >> 5 >> 30 >> 1.0 >> >> >> 5 >> 5 >> 30 >> 1.0 >> >> >> >> Note: we have tested with Hadoop Stream job. >> >> Fair Scheduler Administration Pools PoolRunning JobsMap TasksReduce TasksScheduling >> Mode Min ShareMax ShareRunningFair ShareMin ShareMax ShareRunningFair >> Share research15-90690.05-00.0FAIR tech35-1266690.05-2424.0FAIR default00 >> -00.00-00.0FAIR Running Jobs SubmittedJobIDUserNamePoolPriorityMap TasksReduce >> Tasks FinishedRunningFair ShareWeightFinishedRunningFair ShareWeight Jan >> 16, 08:51 job_201301071639_2118 >> tech streamjob5335328828469969152.jar 30466 / 53724583313.5 1.0 0 / 240 >> 0.0 1.0 Jan 16, 09:56 job_201301071639_2147 >> research streamjob8832181817213433660.jar 4175 / 958190690.0 1.0 0 / 24 >> 00.0 1.0 Jan 16, 10:01 job_201301071639_2148 >> tech streamjob8773848575543653055.jar 1842 / 15484620313.5 1.0 0 / 240 >> 0.0 1.0 Jan 16, 10:08 job_201301071639_2155 >> tech counterfactualsim-prod.eagle-EagleDepthSignalDisabled-prod.eagle 387 >> / 4506363.0 1.0 0 / 242424.0 1.0 >> >> -- >> >> >> >> >> >> >> >> >> > --20cf30781366b3924004d36bc144 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
HI Jeff,

thanks for kindly mail, = I have tested sleep job working=C2=A0pretty=C2=A0good. But we have tested w= ith Hadoop streaming job not proper with fair scheduling=C2=A0Algorithm why= ?. =C2=A0Any other way test=C2=A0Hadoop=C2=A0streaming job, with fair sched= uler=C2=A0

Note:
Tested with RHadoop w= ith rmr.

-Dhanasekaran.

Did I learn something today?= If not, I wasted it.


On Wed, Jan 16, 2013 at 12:02 PM, Jeff B= ean <jwfbean@cloudera.com> wrote:
Validate your scheduler capacity and behavior by using sleep jobs. Submit s= leep jobs to the pools that mirror your production jobs and just check that= the scheduler pool allocation behaves as you expect. The nice thing about = sleep is that you can mimic your real jobs: numbers of tasks and how long t= hey run.

You should be able to determine that the hypothesis posed on this threa= d is correct: that all the slots are taken by other tasks. Indeed, your UI = says that research has 90 running tasks after having completed over 4000, b= ut your emails says no tasks are scheduled. I'm a little confused.=C2= =A0

Jeff


On Wed, Jan 16, 2013 at 8:50 AM, Nan Zhu <zhunans= jtu@gmail.com> wrote:
BTW, what I mentioned is fairsharepreemption =C2=A0not= minimum share=C2=A0

an alternative way to achieve= that is to set minimum share of two queues to be equal(or other allocation= scheme you like), and sum of them is equal to the capacity of the cluster,= and enable minimumSharePreemption

Good Luck!

Best,

--=C2=A0
Nan Zhu
School of Computer Science,
McGill University


=20

On Wednesday, 16= January, 2013 at 11:43 AM, Nan Zhu wrote:

I think you should do that, so that when the allocation= is inconsistent with fair share, the tasks in the queue which occupies mor= e beyond it's fair share will be killed, and the available slots would = be assigned to the other one (assuming the weights of them are the same)

Best,

--=C2=A0
Nan Zhu
School of Computer Science,
McGill University


=20

On Wednesday, 16 January, 2013 a= t 11:32 AM, Dhanasekaran Anbalagan wrote:

HI Nan,

=
We have not enabled=C2=A0Fair Scheduler Preemption.
=

-Dhanasekaran.

Did I learn something today? If not, I wasted i= t.


On Wed, Jan 16, 2013 at 11:21 AM, Nan Zhu &l= t;zhunansjtu@gmai= l.com> wrote:
have you enabled task preemption?

Best,

--=C2=A0
Nan Zhu
School of Computer Science,
McGill University


=20

On Wednesday, 16 January, 2013 a= t 10:45 AM, Justin Workman wrote:

Looks like weight for both pools i= s equal and all map slots are used. Therefore I don't believe anyone ha= s priority for the next slots. Try setting research weight to 2. This shoul= d allow research to take slots as tech released them.=C2=A0

Sent from my iPhone

On Jan 16, 2013, at 8:26 AM, Dhanasek= aran Anbalagan <= bugcy013@gmail.com> wrote:

HI Guys

We configured=C2=A0fair scheduler wit= h cdh4, Fair scheduler not work properly.
Map Task Capacity =3D 1= 380
Reduce Task Capacity =3D 720

We create two users tech and research, we configured eq= ual weight 1 But, I stared job in research user mapper will not allocated w= hy?=C2=A0
please guide me guys.

<?xml version=3D"1.0"?>
<allocations><= /div>
<pool name=3D"tech">
=C2=A0 <minMaps>5</minMaps> =
=C2=A0 <minReduces>5</minReduces>
=C2=A0 <maxRunningJobs>30</maxRunni= ngJobs>
=C2=A0 <weight>1.0</weight>
</pool>
<pool name=3D"research">
=C2=A0 <minMaps>5&= lt;/minMaps>
=C2=A0 <minReduces>5</minReduces>
=C2=A0 <maxRunningJobs>30</maxRunni= ngJobs>
=C2=A0 &l= t;weight>1.0</weight>=C2=A0
</pool>
</allocations>

= Note: we have tested with Hadoop Stream job.

F= air Scheduler Administration

Pools

Map Tasks= <= td>50
PoolRunning JobsReduce TasksScheduling Mode
Min ShareMax ShareRunningFair ShareMin ShareMax ShareRunningFair Share
research15-90690.05-00.0FAIR
tech35-1266690.0-2424.0FAIR
default00-00.0-00.0FAIR

Running Jobs

SubmittedJobIDUserNamePoolPriorityMap TasksReduce Tasks
FinishedRunningFair ShareWeightFini= shedRunningFair ShareWeight
Jan 16, 08:51 job_201301071639_2118tech streamjob5335328828469969152.jar 30466 / 53724583313.5 1.0 0 / 2400.0 1.0
Jan 16, 09:56 job_201301071639_2147research streamjob8832181817213433660.jar 4175 / 958190690.0 1.0 0 / 2400.0 1.0
Jan 16, 10:01 job_201301071639_2148tech streamjob8773848575543653055.jar 1842 / 15484620313.5 1.0 0 / 2400.0 1.0
Jan 16, 10:08 job_201301071639_2155tech counterfactualsim-prod.eagle-EagleDepthSignalDisabled-prod.eagle 387 / 4506363.0 1.0 0 / 242424.0 1.0

--
=C2=A0
=C2=A0
=C2=A0
 =20 =20 =20 =20


=20 =20 =20 =20

=20 =20 =20 =20
=20



--20cf30781366b3924004d36bc144--