hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dongwon Kim <eastcirc...@postech.ac.kr>
Subject Re: Run my own application master on a specific node in a YARN cluster
Date Wed, 01 Apr 2015 06:38:32 GMT
Thanks for your input but I need to launch my own node manager
(different from the Yarn NM) running on each node.
(which is not explained in the original question)

If I were to launch just a single master with a well-known address,
ZooKeeper would be a great solution!

Dongwon Kim

2015-03-31 10:47 GMT+09:00 Drake민영근 <drake.min@nexr.com>:
> Hi,
> In these circumstances, there is no easy way to do that. Maybe use
> workaround. How about using zookeeper for shared storage? The app master
> create predefined zookeeper node when starting with current machine's IP and
> Clients always look for that zookeeper node for app master's location.
> Thanks.
> Drake 민영근 Ph.D
> kt NexR
> On Mon, Mar 30, 2015 at 11:04 AM, Dongwon Kim <eastcirclek@postech.ac.kr>
> wrote:
>> Hello,
>> First of all, I'm using Hadoop-2.6.0. I want to launch my own app
>> master on a specific node in a YARN cluster in order to open a server
>> on a predetermined IP address and port. To that end, I wrote a driver
>> program in which I created a ResourceRequest object and called
>> setResourceName method to set a hostname, and attached it to a
>> ApplicationSubmissionContext object by
>> callingsetAMContainerResourceRequest method.
>> I tried several times but couldn't launch the app master on a specific
>> node. After searching code, I found that RMAppAttemptImpl invalidates
>> what I've set in ResourceRequest as follows:
>>     // Currently, following fields are all hard code,
>>     // TODO: change these fields when we want to support
>>     // priority/resource-name/relax-locality specification for AM
>> containers
>>     // allocation.
>>     appAttempt.amReq.setNumContainers(1);
>>     appAttempt.amReq.setPriority(AM_CONTAINER_PRIORITY);
>>     appAttempt.amReq.setResourceName(ResourceRequest.ANY);
>>     appAttempt.amReq.setRelaxLocality(true);
>> Is there another way to launch a container for an application master
>> on a specific node in Hadoop-2.6.0?
>> Thanks.
>> Dongwon Kim

View raw message