hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bikas Saha <bi...@hortonworks.com>
Subject RE: Container size configuration
Date Tue, 18 Jun 2013 05:13:57 GMT
I think the API allows different size requests at the same priority. The
implementation of the scheduler drops the size information and uses the
last value set. We should probably at least change it to use the largest
value used so that users don't get containers that are too small for them.
YARN-847 tracks this.

Bikas

-----Original Message-----
From: Robert Evans [mailto:evans@yahoo-inc.com]
Sent: Friday, June 14, 2013 7:09 AM
To: yarn-dev@hadoop.apache.org
Subject: Re: Container size configuration

Is this specifically for YARN?  If so, yes you can do this, MR does this
for Maps vs Reduces.  The API right now requires that the different sized
containers have a different priority.

http://hadoop.apache.org/docs/r2.0.5-alpha/hadoop-yarn/hadoop-yarn-site/Wr
i
tingYarnApplications.html

Shows how to make a resource request. It also shows how to make a
AllocateRequest.  If you put in multiple ResourceRequests into the
AllocateRequest it will allocate both of them.  But remember that that the
priority needs to be different, and the priority determines the order in
which the containers will be allocated to your application.

--Bobby

On 6/13/13 10:41 AM, "Yuzhang Han" <yuzhanghan1982@gmail.com> wrote:

>Hi,
>
>I am wondering if I can allocate different size of containers to the
>tasks in a job. For example: Job = <Task1, Task2, Task3>, Task1 = Task2
>= 1024MB, Task3 = 2048MB. How can I achieve this? Many thanks.
>
>Yuzhang

Mime
View raw message