hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Zhuge (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10370) Allow DataNode to be started with numactl
Date Mon, 30 May 2016 22:02:13 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15306974#comment-15306974

John Zhuge commented on HDFS-10370:

Thanks [~dlmarion] for uploading the new patches.

Comments for file {{hdfs}} in {{HDFS-10370-branch-2.004.patch}}.
* 151-158: How about replacing the block with
    if [[ ${HADOOP_DN_NUMACTL_ENABLE == true ]] ; then
        NUMACTL_CMD=$(command -v numactl)
* Line 154: It is discouraged to use {{which}} to detect executable. See explanation in http://stackoverflow.com/questions/592620/check-if-a-program-exists-from-a-bash-script/677212#677212.
* 315 and 317: These 2 lines are almost identical, any way to remove redundancy?
* 317: Where is {{NUMACTL_ARGS}} initialized? Is it supposed to take the value from {{HADOOP_DN_NUMACTL_ARGS}}?
* 317: Inconsistent prefix between {{NUMA_CMD}} and {{NUMACTL_ARGS}}

[~aw] Any generic way to run numactl for Hadoop daemons with different numactl args? How to
suppot JSVC? Maybe using JNI {{set_mempolicy}}?

> Allow DataNode to be started with numactl
> -----------------------------------------
>                 Key: HDFS-10370
>                 URL: https://issues.apache.org/jira/browse/HDFS-10370
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>            Reporter: Dave Marion
>            Assignee: Dave Marion
>         Attachments: HDFS-10370-1.patch, HDFS-10370-2.patch, HDFS-10370-3.patch, HDFS-10370-branch-2.004.patch,
> Allow numactl constraints to be applied to the datanode process. The implementation I
have in mind involves two environment variables (enable and parameters) in the datanode startup
process. Basically, if enabled and numactl exists on the system, then start the java process
using it. Provide a default set of parameters, and allow the user to override the default.
Wiring this up for the non-jsvc use case seems straightforward. Not sure how this can be supported
using jsvc.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message