hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Devaraj k <devara...@huawei.com>
Subject RE: Question related to resource allocation in Yarn!
Date Fri, 06 Sep 2013 05:01:32 GMT
Hi Rahul,

    Could you tell me, what is the version you are using?

·         If you want a container, you need to issue 3 resource requests (1-node local, 1-rack
local and 1-Any(*) ). If you are using 2.1.0-beta<https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&mode=hide&sorter/order=ASC&sorter/field=priority&pid=12313722&customfield_12310320=12324029>
or later versions, you can set the Relax Locality flag to false for getting only on the specified
host.
Can you also share the code how you are requesting for containers…so that we can help you
better..

Thanks
Devaraj k

From: Rahul Bhattacharjee [mailto:rahul.rec.dgp@gmail.com]
Sent: 06 September 2013 09:43
To: user@hadoop.apache.org
Subject: Re: Question related to resource allocation in Yarn!

I could progress a bit on this.

I was not setting responseId while asking for containers.
Still I have one question as why I am only been allocated two containers whereas node manager
can run more containers.

Response while registering the application master -
AM registration response minimumCapability {, memory: 1024, virtual_cores: 1, }, maximumCapability
{, memory: 8192, virtual_cores: 32, },
Thanks,
Rahul

On Thu, Sep 5, 2013 at 8:33 PM, Rahul Bhattacharjee <rahul.rec.dgp@gmail.com<mailto:rahul.rec.dgp@gmail.com>>
wrote:
Hi,
I am trying to make a small poc on top of yarn.
Within the launched application master , I am trying to request for 50 containers and launch
 a same task on those allocated containers.
My config : AM registration response minimumCapability {, memory: 1024, virtual_cores: 1,
}, maximumCapability {, memory: 8192, virtual_cores: 32, },
1) I am asking for 1G mem and 1 core container to the RM. Ideally the RM should return me
6 - 7 containers , but the response always returns with only 2 containers.
Why is that ?
2) So , when in the first ask 2 containers are returned , then I again required the RM for
50 - 2 = 48 containers. I keep getting 0 containers , even if the previously started containers
have finished.
Why is that ?
Any link explaining the allocate request of RM would be very helpful.

Thanks,
Rahul

Mime
View raw message