hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mahadev Konar <maha...@hortonworks.com>
Subject Re: Yarn NodeManager services
Date Tue, 11 Oct 2011 21:28:04 GMT
Shuffle is an auxiliary service in the NM. You might want to take a look at AuxServices.java
in the NodeManager codebase.

Also, you can add config parameters to 
(config knob in NodeManagers)

to add more configurable services in the NodeManager.

Hope that helps.

On Oct 11, 2011, at 2:05 PM, <Milind.Bhandarkar@emc.com> <Milind.Bhandarkar@emc.com>

> Hi,
> As part of MPI implementation in Yarn (aka Hamster), I was looking at
> refactoring some of the functionality into node manager services, so that
> it can be reused by other frameworks too. (Based on the discussion with
> some folks led me to believe that suffle etc is also being planned as NM
> service, separating it from deep integration with node manager.)
> I could not find the documentation for building NM services, in the MR-279
> architecture PDF. Does the service need to reside inside the NM jvm ?  How
> does NM talk to the service ? How do containers discover the service ? (I
> know I can look at the code, but if there is already some docs (or even
> philosophical musings :-), it will make my life easier.
> Thanks,
> - milind
> ---
> Milind Bhandarkar
> Greenplum Labs, EMC
> (Disclaimer: Opinions expressed in this email are those of the author, and
> do not necessarily represent the views of any organization, past or
> present, the author might be affiliated with.)

View raw message