Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Fri, 30 Aug 2013 20:18:52 +0000 (UTC)
From: "Bikas Saha (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12663743.1376519755056.62265.1377893932791@arcas>
In-Reply-To: <JIRA.12663743.1376519755056@arcas>
References: <JIRA.12663743.1376519755056@arcas>
Subject: [jira] [Commented] (YARN-1065) NM should provide AuxillaryService
 data to the container
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/YARN-1065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13755088#comment-13755088 ] 

Bikas Saha commented on YARN-1065:
----------------------------------

There seems to be some misunderstanding here. The NM already provides service metadata in startContainerResponse to the AM.
The main issue is that when the NM actually launches the process of that container then that process has no way to find information about the auxillary services mentioned in the startContainerRequest for that container.
So the desired solution is the following
1) StartContainerRequest sends request for auxillary services to NM (this already happens today)
2) NM sends back auxillary service metadata (say Map<FOO, ByteBuffer>) in the StartContainerResponse (this happens today)
3) When NM launches the process for this container then it should put the service metadata (Map<FOO, ByteBuffer>) in the container environment. Eg. add to environment key=FOO_NM_SERVICE_KEY with value=base64String(ByteBuffer).
4) There should be some helper API to enable the process in the container to retrieve those keys and value. Process uses api, say, Helper.getServiceMetadata(FOO) which looks up FOO_NM_SERVICE_KEY in the env and converts the base64String value back to ByteBuffer. NM could also user Helper.set(FOO, ByteBuffer, env) that sets key=FOO_NM_SERVICE_KEY and base64String(ByteBuffer) into the env. Using helper API will keep both sides consistent.
                
> NM should provide AuxillaryService data to the container
> --------------------------------------------------------
>
>                 Key: YARN-1065
>                 URL: https://issues.apache.org/jira/browse/YARN-1065
>             Project: Hadoop YARN
>          Issue Type: Task
>            Reporter: Bikas Saha
>            Assignee: Xuan Gong
>         Attachments: YARN-1065.1.patch, YARN-1065.2.patch
>
>
> Start container returns auxillary service data to the AM but does not provide the same information to the task itself. It could add that information to the container env with key=service_name and value=service_data. This allows the container to start using the service without having to depend on the AM to send the info to it indirectly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira