ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sumit Mohanty" <smoha...@hortonworks.com>
Subject Re: Review Request 27549: Storm in secure Ambari cluster has config null error when deploying a topology
Date Tue, 04 Nov 2014 00:28:00 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27549/#review59688
-----------------------------------------------------------

Ship it!


Ship It!

- Sumit Mohanty


On Nov. 4, 2014, 12:20 a.m., Alejandro Fernandez wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27549/
> -----------------------------------------------------------
> 
> (Updated Nov. 4, 2014, 12:20 a.m.)
> 
> 
> Review request for Ambari, Mahadev Konar, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-8131
>     https://issues.apache.org/jira/browse/AMBARI-8131
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Create a secure cluster with Storm, kinit as ambari-qa, then attempt to run the following
command to add a topology, which fails,
> 
> ```
> [root@alejandrohdp22-1 tmp]# /usr/hdp/current/storm-client/bin/storm -c java.security.auth.login.config=/etc/storm/conf/client_jaas.conf
jar /usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-*-jar-with-dependencies.jar
storm.starter.ExclamationTopology topo1
> Running: java -client -Dstorm.options=java.security.auth.login.config=/etc/storm/conf/client_jaas.conf
-Dstorm.home=/usr/hdp/2.2.0.0-1744/storm -Dstorm.log.dir=/usr/hdp/2.2.0.0-1744/storm/logs
-Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib
-Dstorm.conf.file= -cp /usr/hdp/2.2.0.0-1744/storm/lib/asm-4.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-security-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/gson-2.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/zookeeper.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clj-stacktrace-0.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/snakeyaml-1.11.jar:/usr/hdp/2.2.0.0-1744/storm/lib/crypto-random-1.2.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-cred-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-lang-2.5.jar:/usr/hdp/2.2.0.0-1744
 /storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.2.0.0-1744/storm/lib/mysql-connector-java.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.cli-0.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/chill-java-0.3.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-http-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/eclipselink-2.5.2-M1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-servlets-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clojure-1.5.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-continuation-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-impl-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-util-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-fileupload-1.2.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-exec-1.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/kryo-2.21.jar:/usr/hdp/2.2.0.0-1744/storm/lib/compojure-1.1.3.jar:/usr/hd
 p/2.2.0.0-1744/storm/lib/javax.persistence-2.1.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-configuration-1.10.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.2.0.0-1744/storm/lib/hadoop-common-2.6.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/logback-core-1.0.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/storm-core-0.9.3.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/disruptor-3.2.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clout-1.0.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/json-simple-1.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.macro-0.1.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/slf4j-api-1.6.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/crypto-equality-1.0.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-anti-forgery-1.0.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-servlet-7.6.13.v
 20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/objenesis-1.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/minlog-1.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clj-time-0.4.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-client-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-server-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-collections-3.2.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.2.0.0-1744/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/guava-11.0.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jersey-bundle-1.17.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/math.numeric-tower-0.0.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/logback-classic-1.0.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/hadoop-auth-2.4.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/co
 mmons-io-2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-audit-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/carbonite-1.4.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-io-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jgrapht-core-0.9.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-storm-plugin-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/joda-time-2.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-logging-1.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-common-0.4.0.2.2.0.0-1744.jar:/usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-0.9.3.2.2.0.0-1744-jar-with-dependencies.jar:/usr/hdp/2.2.0.0-1744/storm/conf:/usr/hdp/2.2.0.0-1744/storm/bin
-Dstorm.jar=/usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-0.9.3.2.2.0.0-1744-jar-with-dependencies.jar
storm.starter.ExclamationTopology topo1
> 773  [main] INFO  backtype.storm.StormSubmitter - Generated ZooKeeper secret payload
for MD5-digest: -5766984969082465146:-6900977625499590902
> 788  [main] INFO  backtype.storm.security.auth.AuthUtils - Got AutoCreds []
> Exception in thread "main" java.lang.RuntimeException: java.security.NoSuchAlgorithmException:
Error constructing implementation (algorithm: JavaLoginConfig, provider: SUN, class: sun.security.provider.ConfigSpiFile)
> 	at backtype.storm.security.auth.AuthUtils.GetConfiguration(AuthUtils.java:66)
> 	at backtype.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:82)
> 	at backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:66)
> 	at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:52)
> 	at backtype.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:36)
> 	at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:211)
> 	at backtype.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:273)
> 	at backtype.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:254)
> 	at storm.starter.ExclamationTopology.main(ExclamationTopology.java:76)
> Caused by: java.security.NoSuchAlgorithmException: Error constructing implementation
(algorithm: JavaLoginConfig, provider: SUN, class: sun.security.provider.ConfigSpiFile)
> 	at java.security.Provider$Service.newInstance(Provider.java:1259)
> 	at sun.security.jca.GetInstance.getInstance(GetInstance.java:243)
> 	at sun.security.jca.GetInstance.getInstance(GetInstance.java:190)
> 	at javax.security.auth.login.Configuration.getInstance(Configuration.java:352)
> 	at backtype.storm.security.auth.AuthUtils.GetConfiguration(AuthUtils.java:64)
> 	... 8 more
> Caused by: java.io.IOException: Configuration Error:
> 	Line 1: expected [{], found [null]
> 	at com.sun.security.auth.login.ConfigFile.match(ConfigFile.java:572)
> 	at com.sun.security.auth.login.ConfigFile.parseLoginEntry(ConfigFile.java:405)
> 	at com.sun.security.auth.login.ConfigFile.readConfig(ConfigFile.java:383)
> 	at com.sun.security.auth.login.ConfigFile.init(ConfigFile.java:283)
> 	at com.sun.security.auth.login.ConfigFile.init(ConfigFile.java:166)
> 	at com.sun.security.auth.login.ConfigFile.<init>(ConfigFile.java:124)
> 	at sun.security.provider.ConfigSpiFile$1.run(ConfigSpiFile.java:72)
> 	at sun.security.provider.ConfigSpiFile$1.run(ConfigSpiFile.java:61)
> #
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at sun.security.provider.ConfigSpiFile.<init>(ConfigSpiFile.java:61)
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> 	at java.security.Provider$Service.newInstance(Provider.java:1254)
> 	... 12 more
> ```
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/params.py
32cb60d 
>   ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/storm.py
5699e57 
>   ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2
cf78af4 
>   ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm.yaml.j2
82ff239 
>   ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm_jaas.conf.j2
7c650df 
> 
> Diff: https://reviews.apache.org/r/27549/diff/
> 
> 
> Testing
> -------
> 
> Ran unit tests on ambari-server,
> ----------------------------------------------------------------------
> Total run:684
> Total errors:0
> Total failures:0
> OK
> 
> 
> Reproduced the issue on a live cluster, and was able to get the topology added.
> E.g.,
> su - ambari-qa
> kinit -kt /etc/security/keytabs/smokeuser.headless.keytab ambari-qa@EXAMPLE.COM
> 
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/params.py
                root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/storm.py
                 root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2
    root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm.yaml.j2
          root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm_jaas.conf.j2
     root@162.216.149.188:/tmp
> scp -i ../gce-key ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/worker-launcher.cfg.j2
 root@162.216.149.188:/tmp
> 
> 
> yes | cp /tmp/params.py               /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/scripts/params.py
> yes | cp /tmp/storm.py                /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/scripts/storm.py
> yes | cp /tmp/client_jaas.conf.j2     /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2
> yes | cp /tmp/storm.yaml.j2           /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/storm.yaml.j2
> yes | cp /tmp/storm_jaas.conf.j2      /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/storm_jaas.conf.j2
> yes | cp /tmp/worker-launcher.cfg.j2  /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/worker-launcher.cfg.j2
> 
> ambari-server restart   (to reload templates)
> 
> Ensure that client_jaas.conf is in /etc/storm/conf/ . Then run,
> /usr/hdp/current/storm-client/bin/storm -c java.security.auth.login.config=/etc/storm/conf/client_jaas.conf
jar /usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-*-jar-with-dependencies.jar
storm.starter.ExclamationTopology topo1
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message