hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aajis...@apache.org
Subject hadoop git commit: HADOOP-12860. Expand section "Data Encryption on HTTP" in SecureMode documentation. Contributed by Wei-Chiu Chuang.
Date Tue, 08 Mar 2016 05:34:29 GMT
Repository: hadoop
Updated Branches:
  refs/heads/trunk c2140d05e -> f86850b54


HADOOP-12860. Expand section "Data Encryption on HTTP" in SecureMode documentation. Contributed
by Wei-Chiu Chuang.


Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/f86850b5
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/f86850b5
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/f86850b5

Branch: refs/heads/trunk
Commit: f86850b544dcb34ee3c9336fad584309e886dbed
Parents: c2140d0
Author: Akira Ajisaka <aajisaka@apache.org>
Authored: Tue Mar 8 14:28:47 2016 +0900
Committer: Akira Ajisaka <aajisaka@apache.org>
Committed: Tue Mar 8 14:28:47 2016 +0900

----------------------------------------------------------------------
 .../src/site/markdown/SecureMode.md             | 27 +++++++++++++-------
 .../src/site/markdown/TimelineServer.md         |  2 +-
 2 files changed, 19 insertions(+), 10 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hadoop/blob/f86850b5/hadoop-common-project/hadoop-common/src/site/markdown/SecureMode.md
----------------------------------------------------------------------
diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/SecureMode.md b/hadoop-common-project/hadoop-common/src/site/markdown/SecureMode.md
index adc3a93..d206b53 100644
--- a/hadoop-common-project/hadoop-common/src/site/markdown/SecureMode.md
+++ b/hadoop-common-project/hadoop-common/src/site/markdown/SecureMode.md
@@ -195,6 +195,13 @@ AES offers the greatest cryptographic strength and the best performance.
At this
 
 Data transfer between Web-console and clients are protected by using SSL(HTTPS). SSL configuration
is recommended but not required to configure Hadoop security with Kerberos.
 
+To enable SSL for web console of HDFS daemons, set `dfs.http.policy` to either `HTTPS_ONLY`
or `HTTP_AND_HTTPS` in hdfs-site.xml.
+Note that this does not affect KMS nor HttpFS, as they are implemented on top of Tomcat and
do not respect this parameter. See [Hadoop KMS](../../hadoop-kms/index.html) and [Hadoop HDFS
over HTTP - Server Setup](../../hadoop-hdfs-httpfs/ServerSetup.html) for instructions on enabling
KMS over HTTPS and HttpFS over HTTPS, respectively.
+
+To enable SSL for web console of YARN daemons, set `yarn.http.policy` to `HTTPS_ONLY` in
yarn-site.xml.
+
+To enable SSL for web console of MapReduce JobHistory server, set `mapreduce.jobhistory.http.policy`
to `HTTPS_ONLY` in mapred-site.xml.
+
 Configuration
 -------------
 
@@ -249,19 +256,18 @@ The following settings allow configuring SSL access to the NameNode
web UI (opti
 | Parameter                    | Value                                           | Notes
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                     |
 |:-----------------------------|:------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | `dfs.http.policy`            | `HTTP_ONLY` or `HTTPS_ONLY` or `HTTP_AND_HTTPS` | `HTTPS_ONLY`
turns off http access. This option takes precedence over the deprecated configuration dfs.https.enable
and hadoop.ssl.enabled. If using SASL to authenticate data transfer protocol instead of running
DataNode as root and using privileged ports, then this property must be set to `HTTPS_ONLY`
to guarantee authentication of HTTP servers. (See `dfs.data.transfer.protection`.) |
-| `dfs.namenode.https-address` | `nn_host_fqdn:50470`                            |      
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                     |
-| `dfs.https.port`             | `50470`                                         |      
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                     |
+| `dfs.namenode.https-address` | `0.0.0.0:50470`                                 | This parameter
is used in non-HA mode and without federation. See [HDFS High Availability](../hadoop-hdfs/HDFSHighAvailabilityWithNFS.html#Deployment)
and [HDFS Federation](../hadoop-hdfs/Federation.html#Federation_Configuration) for details.
                                                                                         
                                                      |
 | `dfs.https.enable`           | `true`                                          | This value
is deprecated. `Use dfs.http.policy`                                                     
                                                                                         
                                                                                         
                                                                                         
                |
 
 ### Secondary NameNode
 
 | Parameter                                                   | Value                   
                | Notes                                                                  
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                                 |
 |:------------------------------------------------------------|:-----------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `dfs.namenode.secondary.http-address`                       | `snn_host_fqdn:50090`   
                |                                                                        
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                                 |
+| `dfs.namenode.secondary.http-address`                       | `0.0.0.0:50090`         
                | HTTP web UI address for the Secondary NameNode.                        
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                                 |
+| `dfs.namenode.secondary.https-address`                      | `0.0.0.0:50091`         
                | HTTPS web UI address for the Secondary NameNode.                       
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                                 |
 | `dfs.secondary.namenode.keytab.file`                        | `/etc/security/keytab/sn.service.keytab`
| Kerberos keytab file for the Secondary NameNode.                                       
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                 |
 | `dfs.secondary.namenode.kerberos.principal`                 | `sn/_HOST@REALM.TLD`    
                | Kerberos principal name for the Secondary NameNode.                    
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                                 |
 | `dfs.secondary.namenode.kerberos.internal.spnego.principal` | `HTTP/_HOST@REALM.TLD`  
                | The server principal used by the Secondary NameNode for web UI SPNEGO authentication.
The SPNEGO server principal begins with the prefix `HTTP/` by convention. If the value is
`'*'`, the web server will attempt to login with every principal specified in the keytab file
`dfs.web.authentication.kerberos.keytab`. For most deployments this can be set to `${dfs.web.authentication.kerberos.principal}`
i.e use the value of `dfs.web.authentication.kerberos.principal`. |
-| `dfs.namenode.secondary.https-port`                         | `50470`                 
                |                                                                        
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                                 |
 
 ### JournalNode
 
@@ -271,6 +277,7 @@ The following settings allow configuring SSL access to the NameNode web
UI (opti
 | `dfs.journalnode.keytab.file`                        | `/etc/security/keytab/jn.service.keytab`
    | Kerberos keytab file for the JournalNode.                                          
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                                                |
 | `dfs.journalnode.kerberos.internal.spnego.principal` | `HTTP/_HOST@REALM.TLD`         
             | The server principal used by the JournalNode for web UI SPNEGO authentication
when Kerberos security is enabled. The SPNEGO server principal begins with the prefix `HTTP/`
by convention. If the value is `'*'`, the web server will attempt to login with every principal
specified in the keytab file `dfs.web.authentication.kerberos.keytab`. For most deployments
this can be set to `${dfs.web.authentication.kerberos.principal}` i.e use the value of `dfs.web.authentication.kerberos.principal`.
|
 | `dfs.web.authentication.kerberos.keytab`             | `/etc/security/keytab/spnego.service.keytab`
| SPNEGO keytab file for the JournalNode. In HA clusters this setting is shared with the Name
Nodes.                                                                                   
                                                                                         
                                                                                         
                                                                                         
                                        |
+| `dfs.journalnode.https-address`                      | `0.0.0.0:8481`                 
             | HTTPS web UI address for the JournalNode.                                 
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                                                         |
 
 ### DataNode
 
@@ -279,7 +286,7 @@ The following settings allow configuring SSL access to the NameNode web
UI (opti
 | `dfs.datanode.data.dir.perm`                     | `700`                              
     |                                                                                   
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                                                                  |
 | `dfs.datanode.address`                           | `0.0.0.0:1004`                     
     | Secure DataNode must use privileged port in order to assure that the server was started
securely. This means that the server must be started via jsvc. Alternatively, this must be
set to a non-privileged port if using SASL to authenticate data transfer protocol. (See `dfs.data.transfer.protection`.)
                                                                                         
                                                                                         
                             |
 | `dfs.datanode.http.address`                      | `0.0.0.0:1006`                     
     | Secure DataNode must use privileged port in order to assure that the server was started
securely. This means that the server must be started via jsvc.                           
                                                                                         
                                                                                         
                                                                                         
                                                             |
-| `dfs.datanode.https.address`                     | `0.0.0.0:50470`                    
     |                                                                                   
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                                                                  |
+| `dfs.datanode.https.address`                     | `0.0.0.0:50475`                    
     | HTTPS web UI address for the Data Node.                                           
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                                                              |
 | `dfs.datanode.kerberos.principal`                | `dn/_HOST@REALM.TLD`               
     | Kerberos principal name for the DataNode.                                         
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                                                                  |
 | `dfs.datanode.keytab.file`                       | `/etc/security/keytab/dn.service.keytab`
| Kerberos keytab file for the DataNode.                                                 
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                                                             |
 | `dfs.encrypt.data.transfer`                      | `false`                            
     | set to `true` when using data encryption                                          
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                                                                  |
@@ -297,10 +304,11 @@ The following settings allow configuring SSL access to the NameNode
web UI (opti
 
 ### ResourceManager
 
-| Parameter                        | Value                                    | Notes   
                                        |
-|:---------------------------------|:-----------------------------------------|:-------------------------------------------------|
-| `yarn.resourcemanager.principal` | `rm/_HOST@REALM.TLD`                     | Kerberos
principal name for the ResourceManager. |
-| `yarn.resourcemanager.keytab`    | `/etc/security/keytab/rm.service.keytab` | Kerberos
keytab file for the ResourceManager.    |
+| Parameter                                    | Value                                  
 | Notes                                                                                 
                                                                                         
                                                                                         
               |
+|:---------------------------------------------|:-----------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `yarn.resourcemanager.principal`             | `rm/_HOST@REALM.TLD`                   
 | Kerberos principal name for the ResourceManager.                                      
                                                                                         
                                                                                         
               |
+| `yarn.resourcemanager.keytab`                | `/etc/security/keytab/rm.service.keytab`
| Kerberos keytab file for the ResourceManager.                                          
                                                                                         
                                                                                         
              |
+| `yarn.resourcemanager.webapp.https.address`  | `${yarn.resourcemanager.hostname}:8090`
 | The https adddress of the RM web application for non-HA. In HA clusters, use `yarn.resourcemanager.webapp.https.address.`*rm-id*
for each ResourceManager. See [ResourceManager High Availability](../../hadoop-yarn/hadoop-yarn-site/ResourceManagerHA.html#Configurations)
for details. |
 
 ### NodeManager
 
@@ -311,6 +319,7 @@ The following settings allow configuring SSL access to the NameNode web
UI (opti
 | `yarn.nodemanager.container-executor.class`       | `org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor`
| Use LinuxContainerExecutor.                             |
 | `yarn.nodemanager.linux-container-executor.group` | `hadoop`                          
                                | Unix group of the NodeManager.                         
|
 | `yarn.nodemanager.linux-container-executor.path`  | `/path/to/bin/container-executor` 
                                | The path to the executable of Linux container executor.
|
+| `yarn.nodemanager.webapp.https.address`           | `0.0.0.0:8044`                    
                                | The https adddress of the NM web application.          
|
 
 ### Configuration for WebAppProxy
 

http://git-wip-us.apache.org/repos/asf/hadoop/blob/f86850b5/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md
----------------------------------------------------------------------
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md
b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md
index b10c860..8ef7d9a 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md
@@ -176,7 +176,7 @@ and cluster operators.
 
 Note that the selection between the HTTP and HTTPS binding is made in the `TimelineClient`
based
 upon the value of the YARN-wide configuration option `yarn.http.policy`; the HTTPS endpoint
will be
-selected if this policy is either of `HTTPS_ONLY` or `HTTP_AND_HTTPS`.
+selected if this policy is `HTTPS_ONLY`.
 
 #### Advanced Server-side configuration
 


Mime
View raw message