hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Seth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1503) Support making additional 'LocalResources' available to running containers
Date Sat, 14 Dec 2013 00:45:07 GMT

    [ https://issues.apache.org/jira/browse/YARN-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13848111#comment-13848111

Siddharth Seth commented on YARN-1503:

bq. A slightly more detailed explanation of the use-case so everyone can understand? And why
something like YARN-1040 is not enough.
YARN-1040 talks about launching multiple processes within the same container. This requirement
is for a single running process - we want to avoid re-launching the process due to the cost
involved with starting a new Java process. The specific use case is running different tasks
within the same JVM - where one task may need some additional jars (Hive UDFs for example).

> Support making additional 'LocalResources' available to running containers
> --------------------------------------------------------------------------
>                 Key: YARN-1503
>                 URL: https://issues.apache.org/jira/browse/YARN-1503
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Siddharth Seth
>            Assignee: Siddharth Seth
> We have a use case, where additional resources (jars, libraries etc) need to be made
available to an already running container. Ideally, we'd like this to be done via YARN (instead
of having potentially multiple containers per node download resources on their own).
> Proposal:
>   NM to support an additional API where a list of resources can be specified. Something
like "localiceResource(ContainerId, Map<String, LocalResource>)
>   NM would also require an additional API to get state for these resources - "getLocalizationState(ContainerId)"
- which returns the current state of all local resources for the specified container(s).

This message was sent by Atlassian JIRA

View raw message