hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karthik Kambatla <ka...@cloudera.com>
Subject Re: Why resource requests are normalized in RM ?
Date Thu, 12 Jun 2014 07:29:26 GMT
I believe they are normalized to be multiples of
yarn.scheduler.increment-allocation-mb.
yarn.scheduler.minimum-allocation-mb can be set to as low as zero. Llama
does this.

As to why normalization, I think it is to make sure there is no external
fragmentation. It is similar to why memory is paged.


On Wed, Jun 11, 2014 at 5:52 PM, Ashwin Shankar <ashwinshankar77@gmail.com>
wrote:

> Hi,
> Anyone knows why resource requests from AMs are normalized to
> be multiples of yarn.scheduler.minimum-allocation-mb which is 1G
> by default ?
> Also is there any problem with reducing yarn.scheduler.minimum-allocation-mb
> to less
> than 1G ?
>
>  /**
>
>    * Utility method to normalize a list of resource requests, by insuring
> that
>
>    * the memory for each request is a multiple of minMemory and is not
> zero.
>
>    */
>
> SchedulerUtils.normalizeRequests()
> --
> Thanks,
> Ashwin
>
>
>

Mime
View raw message