Return-Path: X-Original-To: apmail-hadoop-common-commits-archive@www.apache.org Delivered-To: apmail-hadoop-common-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 86050184FA for ; Sat, 19 Dec 2015 00:55:02 +0000 (UTC) Received: (qmail 48399 invoked by uid 500); 19 Dec 2015 00:55:01 -0000 Delivered-To: apmail-hadoop-common-commits-archive@hadoop.apache.org Received: (qmail 48255 invoked by uid 500); 19 Dec 2015 00:55:01 -0000 Mailing-List: contact common-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-dev@hadoop.apache.org Delivered-To: mailing list common-commits@hadoop.apache.org Received: (qmail 47971 invoked by uid 99); 19 Dec 2015 00:55:01 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 19 Dec 2015 00:55:01 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id C654CE054A; Sat, 19 Dec 2015 00:55:00 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: cnauroth@apache.org To: common-commits@hadoop.apache.org Date: Sat, 19 Dec 2015 00:55:04 -0000 Message-Id: In-Reply-To: References: X-Mailer: ASF-Git Admin Mailer Subject: [5/6] hadoop git commit: HADOOP-12570. HDFS Secure Mode Documentation updates. Contributed by Arpit Agarwal. http://git-wip-us.apache.org/repos/asf/hadoop/blob/d6f1d3b6/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml ---------------------------------------------------------------------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml b/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml index 52037da..a2a1fd2 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml +++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml @@ -1457,13 +1457,137 @@ + dfs.namenode.kerberos.principal + + + The NameNode service principal. This is typically set to + nn/_HOST@REALM.TLD. Each NameNode will subsitute _HOST with its + own fully qualified hostname at startup. The _HOST placeholder + allows using the same configuration setting on both NameNodes + in an HA setup. + + + + + dfs.namenode.keytab.file + + + The keytab file used by each NameNode daemon to login as its + service principal. The principal name is configured with + dfs.namenode.kerberos.principal. + + + + + dfs.datanode.kerberos.principal + + + The DataNode service principal. This is typically set to + dn/_HOST@REALM.TLD. Each DataNode will subsitute _HOST with its + own fully qualified hostname at startup. The _HOST placeholder + allows using the same configuration setting on all DataNodes. + + + + + dfs.datanode.keytab.file + + + The keytab file used by each DataNode daemon to login as its + service principal. The principal name is configured with + dfs.datanode.kerberos.principal. + + + + + dfs.journalnode.kerberos.principal + + + The JournalNode service principal. This is typically set to + jn/_HOST@REALM.TLD. Each JournalNode will subsitute _HOST with its + own fully qualified hostname at startup. The _HOST placeholder + allows using the same configuration setting on all JournalNodes. + + + + + dfs.journalnode.keytab.file + + + The keytab file used by each JournalNode daemon to login as its + service principal. The principal name is configured with + dfs.journalnode.kerberos.principal. + + + + dfs.namenode.kerberos.internal.spnego.principal ${dfs.web.authentication.kerberos.principal} + + The server principal used by the NameNode for web UI SPNEGO + authentication when Kerberos security is enabled. This is + typically set to HTTP/_HOST@REALM.TLD The SPNEGO server principal + begins with the prefix HTTP/ by convention. + + If the value is '*', the web server will attempt to login with + every principal specified in the keytab file + dfs.web.authentication.kerberos.keytab. + + + + + dfs.journalnode.kerberos.internal.spnego.principal + + + The server principal used by the JournalNode HTTP Server for + SPNEGO authentication when Kerberos security is enabled. This is + typically set to HTTP/_HOST@REALM.TLD. The SPNEGO server principal + begins with the prefix HTTP/ by convention. + + If the value is '*', the web server will attempt to login with + every principal specified in the keytab file + dfs.web.authentication.kerberos.keytab. + + For most deployments this can be set to ${dfs.web.authentication.kerberos.principal} + i.e use the value of dfs.web.authentication.kerberos.principal. + dfs.secondary.namenode.kerberos.internal.spnego.principal ${dfs.web.authentication.kerberos.principal} + + The server principal used by the Secondary NameNode for web UI SPNEGO + authentication when Kerberos security is enabled. Like all other + Secondary NameNode settings, it is ignored in an HA setup. + + If the value is '*', the web server will attempt to login with + every principal specified in the keytab file + dfs.web.authentication.kerberos.keytab. + + + + + dfs.web.authentication.kerberos.principal + + + The server principal used by the NameNode for WebHDFS SPNEGO + authentication. + + Required when WebHDFS and security are enabled. In most secure clusters this + setting is also used to specify the values for + dfs.namenode.kerberos.internal.spnego.principal and + dfs.journalnode.kerberos.internal.spnego.principal. + + + + + dfs.web.authentication.kerberos.keytab + + + The keytab file for the principal corresponding to + dfs.web.authentication.kerberos.principal. + http://git-wip-us.apache.org/repos/asf/hadoop/blob/d6f1d3b6/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsMultihoming.md ---------------------------------------------------------------------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsMultihoming.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsMultihoming.md index 0c98d29..4be5511 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsMultihoming.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsMultihoming.md @@ -23,6 +23,8 @@ This document is targetted to cluster administrators deploying `HDFS` in multiho * [Ensuring HDFS Daemons Bind All Interfaces](#Ensuring_HDFS_Daemons_Bind_All_Interfaces) * [Clients use Hostnames when connecting to DataNodes](#Clients_use_Hostnames_when_connecting_to_DataNodes) * [DataNodes use HostNames when connecting to other DataNodes](#DataNodes_use_HostNames_when_connecting_to_other_DataNodes) + * [Multihoming and Hadoop Security](#Multihoming_and_Hadoop_Security) + * [Hostname Lookup](#Hostname_Lookup) Multihoming Background ---------------------- @@ -127,3 +129,43 @@ Rarely, the NameNode-resolved IP address for a DataNode may be unreachable from connecting to other datanodes for data transfer. + +Multihoming and Hadoop Security +------------------------------- + +Configuring multihomed hosts with [Hadoop in Secure Mode](../hadoop-common/SecureMode.html) may require additional configuration. + +### Hostname Lookup + +Kerberos principals for Hadoop Services are specified using the pattern `ServiceName/_HOST@REALM.TLD` e.g. `nn/_HOST@REALM.TLD`. This allows the same configuration file to be used on all hosts. Services will substitute `_HOST` in the principal with their own hostname looked up at runtime. + +When nodes are configured to have multiple hostnames in DNS or in `/etc/hosts` files, a service may lookup a different hostname than what is expected by the server. e.g. intra-cluster traffic between two services may be routed over a private interface but the client service looked up its public hostname. Kerberos authentication will fail since the hostname in the principal does not match the IP address over which the traffic arrived. + +The following setting (available starting Apache Hadoop 2.8.0) can be used to control the hostname looked up the service. + + + hadoop.security.dns.interface + + The name of the Network Interface from which the service should determine + its host name for Kerberos login. e.g. eth2. In a multi-homed environment, + the setting can be used to affect the _HOST subsitution in the service + Kerberos principal. If this configuration value is not set, the service + will use its default hostname as returned by + InetAddress.getLocalHost().getCanonicalHostName(). + + Most clusters will not require this setting. + + + +Services can also be configured to use a specific DNS server for hostname lookups (rarely required). + + + hadoop.security.dns.nameserver + + The host name or IP address of the name server (DNS) which a service Node + should use to determine its own host name for Kerberos Login. Requires + hadoop.security.dns.interface. + + Most clusters will not require this setting. + +