hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Zhuge (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10370) Allow DataNode to be started with numactl
Date Mon, 23 May 2016 22:15:13 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297215#comment-15297215

John Zhuge commented on HDFS-10370:

Could you elaborate a bit more the use cases?

If we are moving into the territory of numa awareness, shall we consider a solution more generic
than just Datanode? As all daemons can be made numa aware.

Do we plan to support membind or cpubind? How to assign daemons to different numa nodes? How
to deal with imbalance in usage? How to monitor numa node stats (numastat(8) output)?

How to support this feature across platforms?

> Allow DataNode to be started with numactl
> -----------------------------------------
>                 Key: HDFS-10370
>                 URL: https://issues.apache.org/jira/browse/HDFS-10370
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>            Reporter: Dave Marion
>            Assignee: Dave Marion
>         Attachments: HDFS-10370-1.patch, HDFS-10370-2.patch, HDFS-10370-3.patch
> Allow numactl constraints to be applied to the datanode process. The implementation I
have in mind involves two environment variables (enable and parameters) in the datanode startup
process. Basically, if enabled and numactl exists on the system, then start the java process
using it. Provide a default set of parameters, and allow the user to override the default.
Wiring this up for the non-jsvc use case seems straightforward. Not sure how this can be supported
using jsvc.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message