hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1065) NM should provide AuxillaryService data to the container
Date Fri, 30 Aug 2013 20:18:52 GMT

    [ https://issues.apache.org/jira/browse/YARN-1065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13755088#comment-13755088

Bikas Saha commented on YARN-1065:

There seems to be some misunderstanding here. The NM already provides service metadata in
startContainerResponse to the AM.
The main issue is that when the NM actually launches the process of that container then that
process has no way to find information about the auxillary services mentioned in the startContainerRequest
for that container.
So the desired solution is the following
1) StartContainerRequest sends request for auxillary services to NM (this already happens
2) NM sends back auxillary service metadata (say Map<FOO, ByteBuffer>) in the StartContainerResponse
(this happens today)
3) When NM launches the process for this container then it should put the service metadata
(Map<FOO, ByteBuffer>) in the container environment. Eg. add to environment key=FOO_NM_SERVICE_KEY
with value=base64String(ByteBuffer).
4) There should be some helper API to enable the process in the container to retrieve those
keys and value. Process uses api, say, Helper.getServiceMetadata(FOO) which looks up FOO_NM_SERVICE_KEY
in the env and converts the base64String value back to ByteBuffer. NM could also user Helper.set(FOO,
ByteBuffer, env) that sets key=FOO_NM_SERVICE_KEY and base64String(ByteBuffer) into the env.
Using helper API will keep both sides consistent.
> NM should provide AuxillaryService data to the container
> --------------------------------------------------------
>                 Key: YARN-1065
>                 URL: https://issues.apache.org/jira/browse/YARN-1065
>             Project: Hadoop YARN
>          Issue Type: Task
>            Reporter: Bikas Saha
>            Assignee: Xuan Gong
>         Attachments: YARN-1065.1.patch, YARN-1065.2.patch
> Start container returns auxillary service data to the AM but does not provide the same
information to the task itself. It could add that information to the container env with key=service_name
and value=service_data. This allows the container to start using the service without having
to depend on the AM to send the info to it indirectly.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message