hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shahab Yunus <shahab.yu...@gmail.com>
Subject Re: number of mappers allowed in a container in hadoop2
Date Wed, 15 Oct 2014 12:47:22 GMT
It cannot run more mappers (tasks) in parallel than the underlying cores
available. Just like it cannot run multiple mappers in parallel if each
mapper's (task's) memory requirements are greater than allocated and
available container size configured on each node.

The links that I provided earlier...see the following section in that one:
Section:"Configuring YARN"

Also this:
http://blog.cloudera.com/blog/2014/04/apache-hadoop-yarn-avoiding-6-time-consuming-gotchas/
Section "1. YARN Concurrency (aka “What Happened to Slots?”)"

This should help in putting things in perspective regarding how resource
allocation for each task, container and resources available on the node
relate to each other.

Regards,
Shahab

On Wed, Oct 15, 2014 at 8:18 AM, SACHINGUPTA <sachin@datametica.com> wrote:

>  but Shahab if i have only 4 core machine then how yarn can run more then
> 4 mappers in parallel
> On Wednesday 15 October 2014 05:45 PM, Shahab Yunus wrote:
>
> It depends on memory settings as well, that how much you want to assign
> resources to each container. Then yarn will run as many mappers in parallel
> as possible.
>
>  See this:
> http://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/
>
> http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.0.6.0/bk_installing_manually_book/content/rpm-chap1-11.html
>
>  Regards,
> Shahab
>
> On Wed, Oct 15, 2014 at 8:09 AM, SACHINGUPTA <sachin@datametica.com>
> wrote:
>
>> Hi guys
>>
>> I have situation in which i have machine with 4 processor and i have 5
>> containers so does it mean i can have only 4 mappers running parallely at a
>> time
>>
>> and number of mappers is not dependent on the number of containers in a
>> machine then what is the use of container concept
>>
>> sorry if i have asked anything obvious.
>>
>> --
>> Thanks
>> Sachin Gupta
>>
>>
>
> --
> Thanks
> Sachin Gupta
>
>

Mime
View raw message