Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of sandeepvreddy@outlook.com
 designates 65.55.111.102 as permitted sender)
Message-ID: <BLU171-W114553BB8BED3E347CC8CE1AC790@phx.gbl>
Content-Type: multipart/alternative;
	boundary="_0ebe23b3-4506-428a-904e-83a992e062f7_"
From: Sandeep L <sandeepvreddy@outlook.com>
To: "user@hadoop.apache.org" <user@hadoop.apache.org>
Subject: RE: Using CapacityScheduler to divide resources between jobs (not
 users)
Date: Tue, 9 Jul 2013 10:30:30 +0530
Importance: Normal
In-Reply-To: 
 <CAAMYKhqX6GCmZRB5O3PnnRR-sFT_0bzd1nq0uxKMb0TRE4MDrw@mail.gmail.com>
References: 
 <CAAMYKhqX6GCmZRB5O3PnnRR-sFT_0bzd1nq0uxKMb0TRE4MDrw@mail.gmail.com>
MIME-Version: 1.0

--_0ebe23b3-4506-428a-904e-83a992e062f7_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

One solution I can suggest is use multiple jobtrackers.Jobtracker1: 2 or 3 =
machines as tasktrackersJobtracker2: around 7 machines as tasktrackersJobtr=
acker3: around 10 machines as tasktrackers
As per requirement you can change number of tasktracker machines and run jo=
bs accordingly.
Thanks=2CSandeep.

Date: Sat=2C 6 Jul 2013 17:12:01 +0300
Subject: Using CapacityScheduler to divide resources between jobs (not user=
s)
From: amits@infolinks.com
To: user@hadoop.apache.org

Hi all=2C=20
I'm running Hadoop 1.0.4 on a modest cluster (~20 machines).The jobs runnin=
g on the cluster can be divided (resource wise) as follows:
=0A=
1. Very short jobs: less then 1 minute.2. Normal jobs: 2-3 minutes up to an=
 hour or two.3. Very long jobs: days of processing. (still not active and t=
he reason for my inquiries here).=0A=

I was thinking of using the CapacityScheduler and divide the cluster resour=
ces so that the long jobs can run without disturbing the other jobs.I read =
that such job queues should be upper bound as well since it may use the ent=
ire cluster resources once it's free but since it takes a long time to fini=
sh=2C it won't release them to other queues as it should. Is it so ?=0A=
Any advise about using the CapacityScheduler in that use case ?
Thanks=2C and sorry for re-sending this message.
Amit. 		 	   		  =

--_0ebe23b3-4506-428a-904e-83a992e062f7_
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<style><!--
.hmmessage P
{
margin:0px=3B
padding:0px
}
body.hmmessage
{
font-size: 12pt=3B
font-family:Calibri
}
--></style></head>
<body class=3D'hmmessage'><div dir=3D'ltr'>One solution I can suggest is us=
e multiple jobtrackers.<div><ol><li>Jobtracker1: 2 or 3 machines as tasktra=
ckers</li><li>Jobtracker2: around 7 machines&nbsp=3Bas tasktrackers</li><li=
>Jobtracker3: around 10 machines&nbsp=3Bas tasktrackers</li></ol><div><br><=
/div><div>As per requirement you can change number of tasktracker machines =
and run jobs accordingly.</div><br>Thanks=2C<div>Sandeep.</div><br><br><div=
><hr id=3D"stopSpelling">Date: Sat=2C 6 Jul 2013 17:12:01 +0300<br>Subject:=
 Using CapacityScheduler to divide resources between jobs (not users)<br>Fr=
om: amits@infolinks.com<br>To: user@hadoop.apache.org<br><br><div dir=3D"lt=
r"><div>Hi all=2C&nbsp=3B</div><div><br></div><div>I'm running Hadoop 1.0.4=
 on a modest cluster (~20 machines).</div><div>The jobs running on the clus=
ter can be divided (resource wise) as follows:</div><div><br></div>=0A=
<div>1. Very short jobs: less then 1 minute.</div><div>2. Normal jobs: 2-3 =
minutes up to an hour or two.</div><div>3. Very long jobs: days of processi=
ng. (still not active and the reason for my inquiries here).</div><div>=0A=
<br></div><div>I was thinking of using the CapacityScheduler and divide the=
 cluster resources so that the long jobs can run without disturbing the oth=
er jobs.</div><div>I read that such job queues should be upper bound as wel=
l since it may use the entire cluster resources once it's free but since it=
 takes a long time to finish=2C it won't release them to other queues as it=
 should. Is it so ?</div>=0A=
<div>Any advise about using the&nbsp=3BCapacityScheduler in that use case ?=
</div><div><br></div><div>Thanks=2C and sorry for re-sending this message.<=
/div><div><br></div><div>Amit.</div></div></div></div> 		 	   		  </div></b=
ody>
</html>=

--_0ebe23b3-4506-428a-904e-83a992e062f7_--