ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hurley" <jhur...@hortonworks.com>
Subject Re: Review Request 30202: RU - Fix hardcoded Namenode address and JMX port
Date Tue, 03 Feb 2015 00:31:40 GMT


> On Jan. 23, 2015, 9:05 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py,
line 84
> > <https://reviews.apache.org/r/30202/diff/2/?file=831417#file831417line84>
> >
> >     This uses url2lib instead of curl; Alerts just got slammed with this when the
environment was kerberized. Without a proper keytab, the URL request would hit a 401 Unauthorized
error.
> >     
> >     Do we use this pattern a lot? I'd think it would fail all over the place.
> 
> Jonathan Hurley wrote:
>     I don't think you fixed this. url2lib won't work when requesting JMX resources in
a kerberized environment.
> 
> Alejandro Fernandez wrote:
>     Jonathan, I ended up testing this on a kerberized cluster and it worked.

I'm really surprised by this. Can you explain to me why it works? url2lib doesn't support
SPNEGO, so even if you had a valid ticket, the library would not be able to negotiate with
the server to use it. I'm not saying that I don't believe you here, but I want to make sure
we understand why it works and that we're not missing something. Since Ambari is running as
root, the ticket would need to be for the root user, which it's definitely not.


- Jonathan


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30202/#review69388
-----------------------------------------------------------


On Feb. 2, 2015, 7:09 p.m., Alejandro Fernandez wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30202/
> -----------------------------------------------------------
> 
> (Updated Feb. 2, 2015, 7:09 p.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Yurii
Shylov.
> 
> 
> Bugs: AMBARI-9289
>     https://issues.apache.org/jira/browse/AMBARI-9289
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Java-side:
> UpgradeHelper hardcodes the JMX port number when figuring out the active and standby
namenodes.
> 
> Python-side:
> When using HA mode, the JMX URLs for NameNode must be taken from dfs.namenode.http-address.<cluster>.<nn#>
> See journalnode_upgrade.py
> E.g.,
> dfs.namenode.http-address.ha.nn1 : c6405.ambari.apache.org:50070
> dfs.namenode.http-address.ha.nn2 : c6403.ambari.apache.org:50070
> 
> 
> Diffs
> -----
> 
>   ambari-common/src/main/python/resource_management/libraries/script/config_dictionary.py
ab05778 
>   ambari-server/src/main/java/org/apache/ambari/server/checks/AbstractCheckDescriptor.java
f49e666 
>   ambari-server/src/main/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheck.java
ba5e804 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
28538c0 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java
239c79c 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 122e0a3

>   ambari-server/src/main/java/org/apache/ambari/server/utils/HTTPUtils.java 38ac643 
>   ambari-server/src/main/java/org/apache/ambari/server/utils/HostAndPort.java PRE-CREATION

>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py
2881c3f 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
7f0971d 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py
PRE-CREATION 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py
bc37240 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params.py
472d684 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py
2634ce8 
>   ambari-server/src/test/java/org/apache/ambari/server/state/CheckHelperTest.java e6cc089

>   ambari-server/src/test/java/org/apache/ambari/server/utils/TestHTTPUtils.java PRE-CREATION

>   ambari-server/src/test/python/stacks/2.0.6/HDFS/test_journalnode.py 2414214 
>   ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-hdfs-secure.json
0686c57 
>   ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-jmx.json 037397a

>   ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-jmx.json
b26df20 
>   ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-status-active.json
PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-namenode-status-standby.json
PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade.json 9a227e7 
> 
> Diff: https://reviews.apache.org/r/30202/diff/
> 
> 
> Testing
> -------
> 
> Verified that both fixes worked in a 3-node HA cluster when performing a Rolling Upgrade.
> The first item is to allow the resolution of the active/standby namendoes to occur, which
is needed by Namenode Prepare and RESTART.
> The second item is for the Journalnode restart to query JMX correctly.
> I also tested with Hbase and ResourceManager.
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 30:14.455s
> [INFO] Finished at: Thu Jan 22 18:11:41 PST 2015
> [INFO] Final Memory: 50M/758M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message