hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj K (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5764) NUMA awareness support for launching containers
Date Mon, 10 Apr 2017 22:42:41 GMT

    [ https://issues.apache.org/jira/browse/YARN-5764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963602#comment-15963602

Devaraj K commented on YARN-5764:

Thanks [~rajesh.balamohan] for taking a look into this.

bq. Was this flag (-XX:useNUMA) enabled in the tasks when running the benchmark?
yes, I used {{-XX:+UseNUMA}} for running the benchmark.

bq. Hive on MR is outdated, network intensive and slow. It would be great, if BB benchmark
can be run with Hive on Tez which optimizes queries to a great extent. It has much better
resource utilization and also elimiates a lot of IO barriers and would be a lot efficient
than MR codebase.
I haven't tried the BB with Hive on Tez, Here we are not evaluating the BB execution engines
performance and I think 'Hive on MR' or any other component would be ok to show case the performance
benefits of NUMA patch.

> NUMA awareness support for launching containers
> -----------------------------------------------
>                 Key: YARN-5764
>                 URL: https://issues.apache.org/jira/browse/YARN-5764
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: nodemanager, yarn
>            Reporter: Olasoji
>            Assignee: Devaraj K
>         Attachments: NUMA Awareness for YARN Containers.pdf, NUMA Performance Results.pdf,
YARN-5764-v0.patch, YARN-5764-v1.patch, YARN-5764-v2.patch
> The purpose of this feature is to improve Hadoop performance by minimizing costly remote
memory accesses on non SMP systems. Yarn containers, on launch, will be pinned to a specific
NUMA node and all subsequent memory allocations will be served by the same node, reducing
remote memory accesses. The current default behavior is to spread memory across all NUMA nodes.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message