hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunil G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5764) NUMA awareness support for launching containers
Date Wed, 11 Jan 2017 17:19:48 GMT

    [ https://issues.apache.org/jira/browse/YARN-5764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818887#comment-15818887

Sunil G commented on YARN-5764:

Thanks [~devaraj.k] for the proposal. Looks very interesting..

As mentioned above, if NM is taking the decision based on cores (NUMA cpus), it ll be more
container specific. Could we apply it more of application specific where few apps containers
only will be NUMA aware. 

Also I think such NUMA aware nodes could be controlled within a specific nodelabel, I think
it may yield better use cases for NUMA. So during NM init, such awareness info could be passed
to RM and it can be made as node attribute. Such nodes could then be labelled together as

> NUMA awareness support for launching containers
> -----------------------------------------------
>                 Key: YARN-5764
>                 URL: https://issues.apache.org/jira/browse/YARN-5764
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: nodemanager, yarn
>            Reporter: Olasoji
>            Assignee: Devaraj K
>         Attachments: NUMA Awareness for YARN Containers.pdf, YARN-5764-v0.patch, YARN-5764-v1.patch
> The purpose of this feature is to improve Hadoop performance by minimizing costly remote
memory accesses on non SMP systems. Yarn containers, on launch, will be pinned to a specific
NUMA node and all subsequent memory allocations will be served by the same node, reducing
remote memory accesses. The current default behavior is to spread memory across all NUMA nodes.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message