hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-13341) Deprecate HADOOP_SERVERNAME_OPTS; replace with (command)_(subcommand)_OPTS
Date Mon, 23 Jan 2017 05:06:27 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-13341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Allen Wittenauer updated HADOOP-13341:
--------------------------------------
    Release Note: 
<!-- markdown -->
Users:
* Ability to set per-command+sub-command options from the command line.
* Makes daemon environment variable options consistent across the project. (See deprecation
list below)
* HADOOP\_CLIENT\_OPTS is now honored for every non-daemon sub-command. Prior to this change,
many sub-commands did not use it.

Developers:
* No longer need to do custom handling for options in the case section of the shell scripts.
* Consolidates all \_OPTS handling into hadoop-functions.sh to enable future projects.
* All daemons running with secure mode features now get \_SECURE\_EXTRA\_OPTS support.

\_OPTS Changes:

| Old | New |
|:---- |:---- |
| HADOOP\_BALANCER\_OPTS | HDFS\_BALANCER\_OPTS | 
| HADOOP\_DATANODE\_OPTS | HDFS\_DATANODE\_OPTS | 
| HADOOP\_DN\_SECURE_EXTRA_OPTS | HDFS\_DATANODE\_SECURE\_EXTRA\_OPTS | 
| HADOOP\_JOB\_HISTORYSERVER\_OPTS | MAPRED\_HISTORYSERVER\_OPTS | 
| HADOOP\_JOURNALNODE\_OPTS | HDFS\_JOURNALNODE\_OPTS | 
| HADOOP\_MOVER\_OPTS | HDFS\_MOVER\_OPTS | 
| HADOOP\_NAMENODE\_OPTS | HDFS\_NAMENODE\_OPTS | 
| HADOOP\_NFS3\_OPTS | HDFS\_NFS3\_OPTS | 
| HADOOP\_NFS3\_SECURE\_EXTRA\_OPTS | HDFS\_NFS3\_SECURE\_EXTRA\_OPTS |
| HADOOP\_PORTMAP\_OPTS | HDFS\_PORTMAP\_OPTS | 
| HADOOP\_SECONDARYNAMENODE\_OPTS | HDFS\_SECONDARYNAMENODE\_OPTS | 
| HADOOP\_ZKFC\_OPTS | HDFS\_ZKFC\_OPTS | 




  was:
<!-- markdown -->
Users:
* Ability to set per-command+sub-command options from the command line.
* Makes daemon environment variable options consistent across the project. (See deprecation
list below)
* HADOOP\_CLIENT\_OPTS is now honored for every non-daemon sub-command. Prior to this change,
many sub-commands did not use it.

Developers:
* No longer need to do custom handling for options in the case section of the shell scripts.
* Consolidates all \_OPTS handling into hadoop-functions.sh to enable future projects.
* All daemons running with secure mode features now get \_SECURE\_EXTRA\_OPTS support.

\_OPTS Changes:

| Old | New |
|:---- |:---- |
| HADOOP\_BALANCER\_OPTS | HDFS\_BALANCER\_OPTS | 
| HADOOP\_DATANODE\_OPTS | HDFS\_DATANODE\_OPTS | 
| HADOOP\_DN\_SECURE_EXTRA_OPTS | HDFS\_DATANODE\_SECURE\_EXTRA\_OPTS | 
| HADOOP\_JOB\_HISTORYSERVER\_OPTS | MAPRED\_HISTORYSERVER\_OPTS | 
| HADOOP\_JOURNALNODE\_OPTS | HDFS\_JOURNALNODE\_OPTS | 
| HADOOP\_MOVER\_OPTS | HDFS\_MOVER\_OPTS | 
| HADOOP\_NAMENODE\_OPTS | HDFS\_NAMENODE\_OPTS | 
| HADOOP\_NFS3\_OPTS | HDFS\_NFS3\_OPTS | 
| HADOOP\_NFS3\_SECURE\_EXTRA\_OPTS | HDFS\_NFS3\_SECURE\_EXTRA\_OPTS | | HADOOP\_PORTMAP\_OPTS
| HDFS\_PORTMAP\_OPTS | 
| HADOOP\_SECONDARYNAMENODE\_OPTS | 
HDFS\_SECONDARYNAMENODE\_OPTS | 
| HADOOP\_ZKFC\_OPTS | HDFS\_ZKFC\_OPTS | 





> Deprecate HADOOP_SERVERNAME_OPTS; replace with (command)_(subcommand)_OPTS
> --------------------------------------------------------------------------
>
>                 Key: HADOOP-13341
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13341
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: scripts
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Allen Wittenauer
>            Assignee: Allen Wittenauer
>             Fix For: 3.0.0-alpha2
>
>         Attachments: HADOOP-13341.00.patch
>
>
> Big features like YARN-2928 demonstrate that even senior level Hadoop developers forget
that daemons need a custom _OPTS env var.  We can replace all of the custom vars with generic
handling just like we do for the username check.
> For example, with generic handling in place:
> || Old Var || New Var ||
> | HADOOP_NAMENODE_OPTS | HDFS_NAMENODE_OPTS |
> | YARN_RESOURCEMANAGER_OPTS | YARN_RESOURCEMANAGER_OPTS |
> | n/a | YARN_TIMELINEREADER_OPTS |
> | n/a | HADOOP_DISTCP_OPTS |
> | n/a | MAPRED_DISTCP_OPTS |
> | HADOOP_DN_SECURE_EXTRA_OPTS | HDFS_DATANODE_SECURE_EXTRA_OPTS |
> | HADOOP_NFS3_SECURE_EXTRA_OPTS | HDFS_NFS3_SECURE_EXTRA_OPTS |
> | HADOOP_JOB_HISTORYSERVER_OPTS | MAPRED_HISTORYSERVER_OPTS |
> This makes it:
> a) consistent across the entire project
> b) consistent for every subcommand
> c) eliminates almost all of the custom appending in the case statements
> It's worth pointing out that subcommands like distcp that sometimes need a higher than
normal client-side heapsize or custom options are a huge win.  Combined with .hadooprc and/or
dynamic subcommands, it means users can easily do customizations based upon their needs without
a lot of weirdo shell aliasing or one line shell scripts off to the side.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message