ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zack Marsh (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AMBARI-12096) Hive Service Check failing after enabling Kerberos
Date Tue, 23 Jun 2015 16:41:01 GMT
Zack Marsh created AMBARI-12096:
-----------------------------------

             Summary: Hive Service Check failing after enabling Kerberos
                 Key: AMBARI-12096
                 URL: https://issues.apache.org/jira/browse/AMBARI-12096
             Project: Ambari
          Issue Type: Bug
         Environment: ambari-2.1.0-1213, hdp-2.3.0.0-2450, sles11sp3

            Reporter: Zack Marsh
            Priority: Critical


After enabling Kerberos, the Hive Service Check is consistently failing.

Errors as seen in Ambari when the Service Check fails:

stderr:
{code}
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/service_check.py",
line 106, in <module>
    HiveServiceCheck().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
line 216, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/service_check.py",
line 102, in service_check
    hcat_service_check()
  File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, in thunk
    return fn(*args, **kwargs)
  File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hcat_service_check.py",
line 65, in hcat_service_check
    logoutput=True)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 157, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 152,
in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 118,
in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line
254, in action_run
    tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call
    tries=tries, try_sleep=try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in
_call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 291, in
_call
    raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt /etc/security/keytabs/smokeuser.headless.keytab
ambari-qa; env JAVA_HOME=/opt/teradata/jvm64/jdk8 /var/lib/ambari-agent/data/tmp/hcatSmoke.sh
hcatsmokeid00270328_date262315 prepare' returned 64. kinit(v5): Client not found in Kerberos
database while getting initial credentials
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: SemanticException MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
Unauthorized connection for super-user: hive/piripiri2.labs.teradata.com@PIRIPIRI.HADOOP.TERADATA.COM
from IP 39.0.40.2)
WARNING: Use "yarn jar" to launch YARN applications.
OK
Time taken: 1.612 seconds
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: SemanticException MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
Unauthorized connection for super-user: hive/piripiri2.labs.teradata.com@PIRIPIRI.HADOOP.TERADATA.COM
from IP 39.0.40.2)
{code}

stdout:
{code}
Test connectivity to hive server
Waiting for the Hive server to start...
2015-06-23 12:26:21,859 - Execute['/usr/bin/kinit -kt /etc/security/keytabs/smokeuser.headless.keytab
ambari-qa-PIRIPIRI@PIRIPIRI.HADOOP.TERADATA.COM; '] {'user': 'ambari-qa'}
2015-06-23 12:26:21,910 - Execute['! beeline -u 'jdbc:hive2://piripiri2.labs.teradata.com:10000/;transportMode=binary;principal=hive/_HOST@PIRIPIRI.HADOOP.TERADATA.COM'
-e '' 2>&1| awk '{print}'|grep -i -e 'Connection refused' -e 'Invalid URL''] {'path':
['/bin/', '/usr/bin/', '/usr/lib/hive/bin/', '/usr/sbin/'], 'user': 'ambari-qa', 'timeout':
30}
Successfully connected to piripiri2.labs.teradata.com on port 10000
Successfully connected to Hive at piripiri3.labs.teradata.com on port 10000 after 3 seconds
2015-06-23 12:26:25,609 - File['/var/lib/ambari-agent/data/tmp/hcatSmoke.sh'] {'content':
StaticFile('hcatSmoke.sh'), 'mode': 0755}
2015-06-23 12:26:25,610 - Writing File['/var/lib/ambari-agent/data/tmp/hcatSmoke.sh'] because
it doesn't exist
2015-06-23 12:26:25,610 - Changing permission for /var/lib/ambari-agent/data/tmp/hcatSmoke.sh
from 644 to 755
2015-06-23 12:26:25,610 - Execute['/usr/bin/kinit -kt /etc/security/keytabs/smokeuser.headless.keytab
ambari-qa; env JAVA_HOME=/opt/teradata/jvm64/jdk8 /var/lib/ambari-agent/data/tmp/hcatSmoke.sh
hcatsmokeid00270328_date262315 prepare'] {'logoutput': True, 'path': ['/usr/sbin', '/usr/local/bin',
'/bin', '/usr/bin', '/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/sbin:/sbin:/usr/lib/ambari-server/*:/sbin:/usr/sbin:/usr/local/sbin:/root/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/opt/teradata/bynet/bin:/opt/teradata/jvm64/jdk8/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:.:/opt/teradata/sm3g/bin:/opt/dell/srvadmin/bin:/opt/dell/srvadmin/sbin:/opt/teradata/dswap/sbin:/usr/tdbms/bin:/opt/teradata/gsctools/bin:/opt/teradata/vmf/bin:/opt/teradata/bin:/var/lib/ambari-agent:/var/lib/ambari-agent:/usr/hdp/current/hive-client/bin:/usr/hdp/current/hadoop-client/bin'],
'tries': 3, 'user': 'ambari-qa', 'try_sleep': 5}
kinit(v5): Client not found in Kerberos database while getting initial credentials
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: SemanticException MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
Unauthorized connection for super-user: hive/piripiri2.labs.teradata.com@PIRIPIRI.HADOOP.TERADATA.COM
from IP 39.0.40.2)
WARNING: Use "yarn jar" to launch YARN applications.
OK
Time taken: 1.448 seconds
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: SemanticException MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
Unauthorized connection for super-user: hive/piripiri2.labs.teradata.com@PIRIPIRI.HADOOP.TERADATA.COM
from IP 39.0.40.2)
2015-06-23 12:26:39,610 - Retrying after 5 seconds. Reason: Execution of '/usr/bin/kinit -kt
/etc/security/keytabs/smokeuser.headless.keytab ambari-qa; env JAVA_HOME=/opt/teradata/jvm64/jdk8
/var/lib/ambari-agent/data/tmp/hcatSmoke.sh hcatsmokeid00270328_date262315 prepare' returned
64. kinit(v5): Client not found in Kerberos database while getting initial credentials
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: SemanticException MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
Unauthorized connection for super-user: hive/piripiri2.labs.teradata.com@PIRIPIRI.HADOOP.TERADATA.COM
from IP 39.0.40.2)
WARNING: Use "yarn jar" to launch YARN applications.
OK
Time taken: 1.448 seconds
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: SemanticException MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
Unauthorized connection for super-user: hive/piripiri2.labs.teradata.com@PIRIPIRI.HADOOP.TERADATA.COM
from IP 39.0.40.2)
kinit(v5): Client not found in Kerberos database while getting initial credentials
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: SemanticException MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
Unauthorized connection for super-user: hive/piripiri2.labs.teradata.com@PIRIPIRI.HADOOP.TERADATA.COM
from IP 39.0.40.2)
WARNING: Use "yarn jar" to launch YARN applications.
OK
Time taken: 1.42 seconds
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: SemanticException MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
Unauthorized connection for super-user: hive/piripiri2.labs.teradata.com@PIRIPIRI.HADOOP.TERADATA.COM
from IP 39.0.40.2)
2015-06-23 12:26:58,283 - Retrying after 5 seconds. Reason: Execution of '/usr/bin/kinit -kt
/etc/security/keytabs/smokeuser.headless.keytab ambari-qa; env JAVA_HOME=/opt/teradata/jvm64/jdk8
/var/lib/ambari-agent/data/tmp/hcatSmoke.sh hcatsmokeid00270328_date262315 prepare' returned
64. kinit(v5): Client not found in Kerberos database while getting initial credentials
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: SemanticException MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
Unauthorized connection for super-user: hive/piripiri2.labs.teradata.com@PIRIPIRI.HADOOP.TERADATA.COM
from IP 39.0.40.2)
WARNING: Use "yarn jar" to launch YARN applications.
OK
Time taken: 1.42 seconds
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: SemanticException MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
Unauthorized connection for super-user: hive/piripiri2.labs.teradata.com@PIRIPIRI.HADOOP.TERADATA.COM
from IP 39.0.40.2)
kinit(v5): Client not found in Kerberos database while getting initial credentials
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: SemanticException MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
Unauthorized connection for super-user: hive/piripiri2.labs.teradata.com@PIRIPIRI.HADOOP.TERADATA.COM
from IP 39.0.40.2)
WARNING: Use "yarn jar" to launch YARN applications.
OK
Time taken: 1.612 seconds
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: SemanticException MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
Unauthorized connection for super-user: hive/piripiri2.labs.teradata.com@PIRIPIRI.HADOOP.TERADATA.COM
from IP 39.0.40.2)
{code}

The cluster has Hive HA enabled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message