ambari-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmitry Lysnichenko (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AMBARI-17258) Storm service check failed after Ambari upgrade
Date Wed, 15 Jun 2016 16:56:09 GMT
Dmitry Lysnichenko created AMBARI-17258:
-------------------------------------------

             Summary: Storm service check failed after Ambari upgrade
                 Key: AMBARI-17258
                 URL: https://issues.apache.org/jira/browse/AMBARI-17258
             Project: Ambari
          Issue Type: Bug
            Reporter: Dmitry Lysnichenko
            Assignee: Dmitry Lysnichenko
         Attachments: AMBARI-17258.patch


Marked this as blocker since the issue affected many EU End to End test runs that involved
running a service check as part of pre-check condition for EU

ambari-server-2.4.0.0-665.x86_64
ambari-server --hash

*Steps*
# Deploy HDP-2.3.2.0 cluster with Ambari 2.1.2 (secure, non-HA cluster)
# Upgrade Ambari to 2.4.0.0-665
# Stop and Start all services
# Run service check for Storm

*Result*
Failure as below:
{code}
stderr:   /var/lib/ambari-agent/data/errors-729.txt

Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/STORM/0.9.1/package/scripts/service_check.py",
line 79, in <module>
ServiceCheck().execute()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line
257, in execute
method(env)
File "/var/lib/ambari-agent/cache/common-services/STORM/0.9.1/package/scripts/service_check.py",
line 70, in service_check
user=params.storm_user
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160,
in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124,
in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line
273, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner
result = function(command, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call
tries=tries, try_sleep=try_sleep)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 293, in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of 'storm jar /tmp/wordCount.jar storm.starter.WordCountTopology
WordCountid16ac396c_date241516' returned 1. 951  [main] INFO  b.s.u.Utils - Using defaults.yaml
from resources
1088 [main] INFO  b.s.u.Utils - Using storm.yaml from resources
1184 [main] INFO  b.s.u.Utils - Using defaults.yaml from resources
1240 [main] INFO  b.s.u.Utils - Using storm.yaml from resources
1285 [main] INFO  b.s.StormSubmitter - Generated ZooKeeper secret payload for MD5-digest:
-7463484230184273671:-7128399429695564111
1287 [main] INFO  b.s.s.a.AuthUtils - Got AutoCreds []
1292 [main] WARN  b.s.u.NimbusClient - Using deprecated config nimbus.host for backward compatibility.
Please update your storm.yaml so it only has config nimbus.seeds
1352 [main] INFO  b.s.u.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs [2000] the
maxSleepTimeMs [60000] the maxRetries [5]
1615 [main] INFO  o.a.s.z.Login - successfully logged in.
1650 [main] ERROR o.a.t.t.TSaslTransport - SASL negotiation failure
javax.security.sasl.SaslException: GSS initiate failed
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212) ~[?:1.7.0_67]
at org.apache.thrift7.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94)
~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at org.apache.thrift7.transport.TSaslTransport.open(TSaslTransport.java:271) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at org.apache.thrift7.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:145)
[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:141)
[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.7.0_67]
at javax.security.auth.Subject.doAs(Subject.java:415) [?:1.7.0_67]
at backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin.connect(KerberosSaslTransportPlugin.java:140)
[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:48)
[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:103) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:72) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:104) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:95) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:67) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:225) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:271) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:157) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at storm.starter.WordCountTopology.main(WordCountTopology.java:77) [wordCount.jar:?]
Caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism level: Server
not found in Kerberos database (7) - UNKNOWN_SERVER)
at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710) ~[?:1.7.0_67]
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248) ~[?:1.7.0_67]
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) ~[?:1.7.0_67]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193) ~[?:1.7.0_67]
... 18 more
Caused by: sun.security.krb5.KrbException: Server not found in Kerberos database (7) - UNKNOWN_SERVER
at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:73) ~[?:1.7.0_67]
at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:192) ~[?:1.7.0_67]
at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:203) ~[?:1.7.0_67]
at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:309) ~[?:1.7.0_67]
at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:115)
~[?:1.7.0_67]
at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:454) ~[?:1.7.0_67]
at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:641) ~[?:1.7.0_67]
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248) ~[?:1.7.0_67]
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) ~[?:1.7.0_67]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193) ~[?:1.7.0_67]
... 18 more
Caused by: sun.security.krb5.Asn1Exception: Identifier doesn't match expected value (906)
at sun.security.krb5.internal.KDCRep.init(KDCRep.java:143) ~[?:1.7.0_67]
at sun.security.krb5.internal.TGSRep.init(TGSRep.java:66) ~[?:1.7.0_67]
at sun.security.krb5.internal.TGSRep.<init>(TGSRep.java:61) ~[?:1.7.0_67]
at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:55) ~[?:1.7.0_67]
at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:192) ~[?:1.7.0_67]
at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:203) ~[?:1.7.0_67]
at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:309) ~[?:1.7.0_67]
at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:115)
~[?:1.7.0_67]
at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:454) ~[?:1.7.0_67]
at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:641) ~[?:1.7.0_67]
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248) ~[?:1.7.0_67]
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) ~[?:1.7.0_67]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193) ~[?:1.7.0_67]
... 18 more
1666 [main] ERROR b.s.s.a.k.KerberosSaslTransportPlugin - Client failed to open SaslClientTransport
to interact with a server during session initiation: org.apache.thrift7.transport.TTransportException:
GSS initiate failed
org.apache.thrift7.transport.TTransportException: GSS initiate failed
at org.apache.thrift7.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at org.apache.thrift7.transport.TSaslTransport.open(TSaslTransport.java:316) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at org.apache.thrift7.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:145)
[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:141)
[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.7.0_67]
at javax.security.auth.Subject.doAs(Subject.java:415) [?:1.7.0_67]
at backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin.connect(KerberosSaslTransportPlugin.java:140)
[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:48)
[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:103) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:72) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:104) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:95) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:67) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:225) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:271) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:157) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at storm.starter.WordCountTopology.main(WordCountTopology.java:77) [wordCount.jar:?]
1675 [main] WARN  b.s.u.NimbusClient - Ignoring exception while trying to get leader nimbus
info from localhost. will retry with a different seed host.
org.apache.thrift7.transport.TTransportException: SASL authentication not complete
at org.apache.thrift7.transport.TSaslTransport.write(TSaslTransport.java:474) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at org.apache.thrift7.transport.TSaslClientTransport.write(TSaslClientTransport.java:37) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at org.apache.thrift7.protocol.TBinaryProtocol.writeI32(TBinaryProtocol.java:178) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at org.apache.thrift7.protocol.TBinaryProtocol.writeMessageBegin(TBinaryProtocol.java:106)
~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at org.apache.thrift7.TServiceClient.sendBase(TServiceClient.java:62) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.generated.Nimbus$Client.send_getClusterInfo(Nimbus.java:553) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.generated.Nimbus$Client.getClusterInfo(Nimbus.java:546) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:68) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:225) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:271) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:157) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at storm.starter.WordCountTopology.main(WordCountTopology.java:77) [wordCount.jar:?]
Exception in thread "main" java.lang.RuntimeException: Could not find leader nimbus from seed
hosts [localhost]. Did you specify a valid list of nimbus hosts for config nimbus.seeds
at backtype.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:90)
at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:225)
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:271)
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:157)
at storm.starter.WordCountTopology.main(WordCountTopology.java:77)
stdout:   /var/lib/ambari-agent/data/output-729.txt

2016-06-15 06:24:45,500 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2016-06-15 06:24:45,503 - checked_call['hostid'] {}
2016-06-15 06:24:45,516 - checked_call returned (0, '16ac396c')
2016-06-15 06:24:45,517 - File['/tmp/wordCount.jar'] {'owner': 'storm', 'content': StaticFile('wordCount.jar')}
2016-06-15 06:24:45,526 - Execute['storm jar /tmp/wordCount.jar storm.starter.WordCountTopology
WordCountid16ac396c_date241516'] {'logoutput': True, 'path': ['/usr/hdp/current/storm-client/bin'],
'user': 'storm'}
951  [main] INFO  b.s.u.Utils - Using defaults.yaml from resources
1088 [main] INFO  b.s.u.Utils - Using storm.yaml from resources
1184 [main] INFO  b.s.u.Utils - Using defaults.yaml from resources
1240 [main] INFO  b.s.u.Utils - Using storm.yaml from resources
1285 [main] INFO  b.s.StormSubmitter - Generated ZooKeeper secret payload for MD5-digest:
-7463484230184273671:-7128399429695564111
1287 [main] INFO  b.s.s.a.AuthUtils - Got AutoCreds []
1292 [main] WARN  b.s.u.NimbusClient - Using deprecated config nimbus.host for backward compatibility.
Please update your storm.yaml so it only has config nimbus.seeds
1352 [main] INFO  b.s.u.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs [2000] the
maxSleepTimeMs [60000] the maxRetries [5]
1615 [main] INFO  o.a.s.z.Login - successfully logged in.
1650 [main] ERROR o.a.t.t.TSaslTransport - SASL negotiation failure
javax.security.sasl.SaslException: GSS initiate failed
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212) ~[?:1.7.0_67]
at org.apache.thrift7.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94)
~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at org.apache.thrift7.transport.TSaslTransport.open(TSaslTransport.java:271) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at org.apache.thrift7.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:145)
[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:141)
[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.7.0_67]
at javax.security.auth.Subject.doAs(Subject.java:415) [?:1.7.0_67]
at backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin.connect(KerberosSaslTransportPlugin.java:140)
[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:48)
[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:103) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:72) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:104) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:95) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:67) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:225) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:271) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:157) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at storm.starter.WordCountTopology.main(WordCountTopology.java:77) [wordCount.jar:?]
Caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism level: Server
not found in Kerberos database (7) - UNKNOWN_SERVER)
at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710) ~[?:1.7.0_67]
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248) ~[?:1.7.0_67]
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) ~[?:1.7.0_67]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193) ~[?:1.7.0_67]
... 18 more
Caused by: sun.security.krb5.KrbException: Server not found in Kerberos database (7) - UNKNOWN_SERVER
at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:73) ~[?:1.7.0_67]
at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:192) ~[?:1.7.0_67]
at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:203) ~[?:1.7.0_67]
at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:309) ~[?:1.7.0_67]
at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:115)
~[?:1.7.0_67]
at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:454) ~[?:1.7.0_67]
at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:641) ~[?:1.7.0_67]
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248) ~[?:1.7.0_67]
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) ~[?:1.7.0_67]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193) ~[?:1.7.0_67]
... 18 more
Caused by: sun.security.krb5.Asn1Exception: Identifier doesn't match expected value (906)
at sun.security.krb5.internal.KDCRep.init(KDCRep.java:143) ~[?:1.7.0_67]
at sun.security.krb5.internal.TGSRep.init(TGSRep.java:66) ~[?:1.7.0_67]
at sun.security.krb5.internal.TGSRep.<init>(TGSRep.java:61) ~[?:1.7.0_67]
at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:55) ~[?:1.7.0_67]
at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:192) ~[?:1.7.0_67]
at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:203) ~[?:1.7.0_67]
at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:309) ~[?:1.7.0_67]
at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:115)
~[?:1.7.0_67]
at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:454) ~[?:1.7.0_67]
at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:641) ~[?:1.7.0_67]
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248) ~[?:1.7.0_67]
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) ~[?:1.7.0_67]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193) ~[?:1.7.0_67]
... 18 more
1666 [main] ERROR b.s.s.a.k.KerberosSaslTransportPlugin - Client failed to open SaslClientTransport
to interact with a server during session initiation: org.apache.thrift7.transport.TTransportException:
GSS initiate failed
org.apache.thrift7.transport.TTransportException: GSS initiate failed
at org.apache.thrift7.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at org.apache.thrift7.transport.TSaslTransport.open(TSaslTransport.java:316) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at org.apache.thrift7.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:145)
[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:141)
[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.7.0_67]
at javax.security.auth.Subject.doAs(Subject.java:415) [?:1.7.0_67]
at backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin.connect(KerberosSaslTransportPlugin.java:140)
[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:48)
[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:103) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:72) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:104) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:95) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:67) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:225) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:271) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:157) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at storm.starter.WordCountTopology.main(WordCountTopology.java:77) [wordCount.jar:?]
1675 [main] WARN  b.s.u.NimbusClient - Ignoring exception while trying to get leader nimbus
info from localhost. will retry with a different seed host.
org.apache.thrift7.transport.TTransportException: SASL authentication not complete
at org.apache.thrift7.transport.TSaslTransport.write(TSaslTransport.java:474) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at org.apache.thrift7.transport.TSaslClientTransport.write(TSaslClientTransport.java:37) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at org.apache.thrift7.protocol.TBinaryProtocol.writeI32(TBinaryProtocol.java:178) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at org.apache.thrift7.protocol.TBinaryProtocol.writeMessageBegin(TBinaryProtocol.java:106)
~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at org.apache.thrift7.TServiceClient.sendBase(TServiceClient.java:62) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.generated.Nimbus$Client.send_getClusterInfo(Nimbus.java:553) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.generated.Nimbus$Client.getClusterInfo(Nimbus.java:546) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:68) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:225) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:271) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:157) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
at storm.starter.WordCountTopology.main(WordCountTopology.java:77) [wordCount.jar:?]
Exception in thread "main" java.lang.RuntimeException: Could not find leader nimbus from seed
hosts [localhost]. Did you specify a valid list of nimbus hosts for config nimbus.seeds
at backtype.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:90)
at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:225)
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:271)
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:157)
at storm.starter.WordCountTopology.main(WordCountTopology.java:77)

Command failed after 1 tries
{code}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message