Mailing-List: contact dev-help@ambari.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ambari.apache.org
Content-Type: multipart/alternative;
 boundary="===============6012986787288101920=="
MIME-Version: 1.0
Subject: Re: Review Request 30202: RU - Fix hardcoded Namenode address and JMX
 port
From: "Alejandro Fernandez" <afernandez@hortonworks.com>
To: "Nate Cole" <ncole@hortonworks.com>,
 "Dmitro Lisnichenko" <dlysnichenko@hortonworks.com>,
 "Yurii Shylov" <yurii.shylov@gmail.com>,
 "Jonathan Hurley" <jhurley@hortonworks.com>
Cc: "Ambari" <dev@ambari.apache.org>,
 "Alejandro Fernandez" <afernandez@hortonworks.com>
Date: Tue, 03 Feb 2015 01:09:01 -0000
Message-ID: <20150203010901.30543.32288@reviews.apache.org>
Auto-Submitted: auto-generated
Sender: "Alejandro Fernandez" <noreply@reviews.apache.org>
References: <20150203002844.30543.25561@reviews.apache.org>
In-Reply-To: <20150203002844.30543.25561@reviews.apache.org>
Reply-To: "Alejandro Fernandez" <afernandez@hortonworks.com>

--===============6012986787288101920==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit


> On Feb. 3, 2015, 12:28 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py, line 82
> > <https://reviews.apache.org/r/30202/diff/4/?file=844619#file844619line82>
> >
> >     I'm surprised that this works. Can we always be certain that the bare name of the hdfs principal is an actual user?
> 
> Alejandro Fernandez wrote:
>     In a kerberized environment, it will be hdfs@REAL.COM, which isn't actually a system user.
> 
> Jonathan Hurley wrote:
>     Right, but you're stripping off the `hdfs` part. What if it's `foo@REAL.com`; are we guaranteed that we can execute commands as the `foo` user?

So I checked with Dilli from the security team. The keytab file access is a principal@REALM, and the principal must exist as a user on the box.
The python code first tries to sudo su {user}, and then run the kinit command. Technically, we can use either, but it's clearer if we run the command as the regular hdfs_user. I'll make this change right now.


- Alejandro


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30202/#review70653
-----------------------------------------------------------


On Feb. 3, 2015, 12:38 a.m., Alejandro Fernandez wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30202/
> -----------------------------------------------------------
> 
> (Updated Feb. 3, 2015, 12:38 a.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Yurii Shylov.
> 
> 
> Bugs: AMBARI-9289
>     https://issues.apache.org/jira/browse/AMBARI-9289
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Java-side:
> UpgradeHelper hardcodes the JMX port number when figuring out the active and standby namenodes.
> 
> Python-side:
> When using HA mode, the JMX URLs for NameNode must be taken from dfs.namenode.http-address.<cluster>.<nn#>
> See journalnode_upgrade.py
> E.g.,
> dfs.namenode.http-address.ha.nn1 : c6405.ambari.apache.org:50070
> dfs.namenode.http-address.ha.nn2 : c6403.ambari.apache.org:50070
> 
> 
> Diffs
> -----
> 
>   ambari-common/src/main/python/resource_management/libraries/script/config_dictionary.py ab05778 
>   ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java f49e666 
>   ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java ba5e804 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java 28538c0 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java 239c79c 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 122e0a3 
>   ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 38ac643 
>   ambari-server/src/main/java/org/apache/ambari/server/utils/HostAndPort.java PRE-CREATION 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py 2881c3f 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 7f0971d 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py PRE-CREATION 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py bc37240 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params.py 472d684 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 2634ce8 
>   ambari-server/src/test/java/org/apache/ambari/server/state/CheckHelperTest.java e6cc089 
>   ambari-server/src/test/java/org/apache/ambari/server/utils/TestHTTPUtils.java PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.0.6/HDFS/test_journalnode.py 2414214 
>   ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-hdfs-secure.json 0686c57 
>   ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-jmx.json 037397a 
>   ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-jmx.json b26df20 
>   ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-status-active.json PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-status-standby.json PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade.json 9a227e7 
> 
> Diff: https://reviews.apache.org/r/30202/diff/
> 
> 
> Testing
> -------
> 
> Verified that both fixes worked in a 3-node HA cluster when performing a Rolling Upgrade.
> The first item is to allow the resolution of the active/standby namendoes to occur, which is needed by Namenode Prepare and RESTART.
> The second item is for the Journalnode restart to query JMX correctly.
> I also tested with Hbase and ResourceManager.
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 30:14.455s
> [INFO] Finished at: Thu Jan 22 18:11:41 PST 2015
> [INFO] Final Memory: 50M/758M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>


--===============6012986787288101920==--