hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ricardo Fajardo <>
Subject Re: Pls Help me - Hive Kerberos Issue
Date Mon, 30 Jan 2017 15:57:09 GMT

1. klist -fe

[cloudera@quickstart bin]$ klist -fe
Ticket cache: FILE:/tmp/krb5cc_501
Default principal: t_fajar@ADS.AUTODESK.COM

Valid starting     Expires            Service principal
01/30/17 10:52:37  01/30/17 20:52:43  krbtgt/ADS.AUTODESK.COM@ADS.AUTODESK.COM
renew until 01/31/17 10:52:37, Flags: FPRIA
Etype (skey, tkt): arcfour-hmac, arcfour-hmac
[cloudera@quickstart bin]$

2. relevant entries from HiveServer2 log

beeline> !connect jdbc:hive2://localhost:10000/default;principal=hive/_HOST@ADS.AUTODESK.COM;hive.server2.proxy.user=t_fajar
!connect jdbc:hive2://localhost:10000/default;principal=hive/_HOST@ADS.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/cloudera/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.6.2/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/cloudera/.m2/repository/org/slf4j/slf4j-log4j12/1.6.1/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/cloudera/.m2/repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Connecting to jdbc:hive2://localhost:10000/default;principal=hive/_HOST@ADS.AUTODESK.COM;hive.server2.proxy.user=t_fajar
17/01/27 16:16:36 INFO Utils: Supplied authorities: localhost:10000
17/01/27 16:16:36 INFO Utils: Resolved authority: localhost:10000
17/01/27 16:16:36 DEBUG MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate$UgiMetrics.loginSuccess with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time,
value=[Rate of successful kerberos logins and latency (milliseconds)], about=, type=DEFAULT,
always=false, sampleName=Ops)
17/01/27 16:16:36 DEBUG MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate$UgiMetrics.loginFailure with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time,
value=[Rate of failed kerberos logins and latency (milliseconds)], about=, type=DEFAULT, always=false,
17/01/27 16:16:36 DEBUG MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate$UgiMetrics.getGroups with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time,
value=[GetGroups], about=, type=DEFAULT, always=false, sampleName=Ops)
17/01/27 16:16:36 DEBUG MetricsSystemImpl: UgiMetrics, User and group related metrics
17/01/27 16:16:37 DEBUG Shell: setsid exited with exit code 0
17/01/27 16:16:37 DEBUG Groups:  Creating new Groups object
17/01/27 16:16:37 DEBUG NativeCodeLoader: Trying to load the custom-built native-hadoop library...
17/01/27 16:16:37 DEBUG NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError:
no hadoop in java.library.path
17/01/27 16:16:37 DEBUG NativeCodeLoader: java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
17/01/27 16:16:37 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform...
using builtin-java classes where applicable
17/01/27 16:16:37 DEBUG PerformanceAdvisory: Falling back to shell based
17/01/27 16:16:37 DEBUG JniBasedUnixGroupsMappingWithFallback: Group mapping
17/01/27 16:16:38 DEBUG Groups: Group mapping;
cacheTimeout=300000; warningDeltaMs=5000
17/01/27 16:16:38 DEBUG UserGroupInformation: hadoop login
17/01/27 16:16:38 DEBUG UserGroupInformation: hadoop login commit
17/01/27 16:16:38 DEBUG UserGroupInformation: using local user:UnixPrincipal: cloudera
17/01/27 16:16:38 DEBUG UserGroupInformation: Using user: "UnixPrincipal: cloudera" with name
17/01/27 16:16:38 DEBUG UserGroupInformation: User entry: "cloudera"
17/01/27 16:16:56 DEBUG UserGroupInformation: UGI loginUser:cloudera (auth:SIMPLE)
17/01/27 16:16:56 DEBUG HadoopThriftAuthBridge: Current authMethod = SIMPLE
17/01/27 16:16:56 DEBUG HadoopThriftAuthBridge: Setting UGI conf as passed-in authMethod of
kerberos != current.
17/01/30 10:24:45 DEBUG UserGroupInformation: PrivilegedAction as:cloudera (auth:SIMPLE) from:org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Client.createClientTransport(
17/01/30 10:55:02 DEBUG UserGroupInformation: PrivilegedAction as:cloudera (auth:SIMPLE)
17/01/30 10:55:02 DEBUG TSaslTransport: opening transport org.apache.thrift.transport.TSaslClientTransport@1119f7c5
17/01/30 10:55:02 ERROR TSaslTransport: SASL negotiation failure GSS initiate failed
at ~[?:1.7.0_67]
at org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(
at [libthrift-0.9.3.jar:0.9.3]
at [libthrift-0.9.3.jar:0.9.3]
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$
at Method) ~[?:1.7.0_67]
at [?:1.7.0_67]
at [hadoop-common-2.7.2.jar:?]
at org.apache.hive.jdbc.HiveConnection.openTransport( [classes/:?]
at org.apache.hive.jdbc.HiveConnection.<init>( [classes/:?]
at org.apache.hive.jdbc.HiveDriver.connect( [classes/:?]
at java.sql.DriverManager.getConnection( [?:1.7.0_67]
at java.sql.DriverManager.getConnection( [?:1.7.0_67]
at org.apache.hive.beeline.DatabaseConnection.connect( [classes/:?]
at org.apache.hive.beeline.DatabaseConnection.getConnection( [classes/:?]
at org.apache.hive.beeline.Commands.connect( [classes/:?]
at org.apache.hive.beeline.Commands.connect( [classes/:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_67]
at sun.reflect.NativeMethodAccessorImpl.invoke( ~[?:1.7.0_67]
at sun.reflect.DelegatingMethodAccessorImpl.invoke( ~[?:1.7.0_67]
at java.lang.reflect.Method.invoke( ~[?:1.7.0_67]
at org.apache.hive.beeline.ReflectiveCommandHandler.execute(
at org.apache.hive.beeline.BeeLine.execCommandWithPrefix( [classes/:?]
at org.apache.hive.beeline.BeeLine.dispatch( [classes/:?]
at org.apache.hive.beeline.BeeLine.execute( [classes/:?]
at org.apache.hive.beeline.BeeLine.begin( [classes/:?]
at org.apache.hive.beeline.BeeLine.mainWithInputRedirection( [classes/:?]
at org.apache.hive.beeline.BeeLine.main( [classes/:?]
Caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism level: Failed
to find any Kerberos tgt)
at ~[?:1.7.0_67]
at ~[?:1.7.0_67]
at ~[?:1.7.0_67]
at ~[?:1.7.0_67]
at ~[?:1.7.0_67]
at ~[?:1.7.0_67]
at ~[?:1.7.0_67]
... 29 more
17/01/30 10:55:02 DEBUG TSaslTransport: CLIENT: Writing message with status BAD and payload
length 19
17/01/30 10:55:02 WARN HiveConnection: Failed to connect to localhost:10000
HS2 may be unavailable, check server status
Error: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:10000/default;principal=hive/_HOST@ADS.AUTODESK.COM;hive.server2.proxy.user=t_fajar:
GSS initiate failed (state=08S01,code=0)

From: Vivek Shrivastava <>
Sent: Monday, January 30, 2017 10:48:35 AM
Subject: Re: Pls Help me - Hive Kerberos Issue

Please paste the output of
1. klist -fe
2. relevant entries from HiveServer2 log

On Mon, Jan 30, 2017 at 10:11 AM, Ricardo Fajardo <<>>

I could not resolve the problem.

I have debugged the code and I found out that:

1. On the org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge class   line 208


UserGroupInformation.getCurrentUser return (). Two (....


This method always returns the user of the operative system but and I need authenticate the
user set on the property: hive.server2.proxy.user=yourid because I have a token for this one.

2. I have found out that the hive.server2.proxy.user is implemented on the org.apache.hive.jdbc.HiveConnection
class method: openSession() but this code is never executed.

3. On the org.apache.hive.service.auth.HiveAuthFactory class there is this code on the method


      if (authTypeStr.equalsIgnoreCase(AuthTypes.KERBEROS.getAuthName())) {
        // no-op

It means that Kerberos authentication is not implemented?

Please anyone can help me??



From: Dulam, Naresh <<>>
Sent: Thursday, January 26, 2017 8:41:48 AM
Subject: RE: Pls Help me - Hive Kerberos Issue

Kinit   yourid -k -t your.keytab yourid@MY-REALM.COM<mailto:yourid@MY-REALM.COM>

# Connect using following JDBC connection string
# jdbc:hive2://;principal=hive/_HOST@MY-REALM.COM;hive.server2.proxy.user=yourid<;principal=hive/_HOST@MY-REALM.COM;hive.server2.proxy.user=yourid>

From: Ricardo Fajardo [<>]
Sent: Thursday, January 26, 2017 1:37 AM
Subject: Pls Help me - Hive Kerberos Issue


Please I need your help with the Kerberos authentication with Hive.

I am following this guide:

But I am getting this error:

Caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism level: Failed
to find any Kerberos tgt)

I have a remote Kerberos server and I can generate a token with kinit for my user. I created
a keytab file with my passwd for my user. Please tell me if it is ok.

On the another hand when I am debugging the hive code the operative system user is authenticated
but I need authenticate my Kerberos user, can you tell me how I can achieve that? How can
I store my tickets where Hive can load it?? or How can I verify where Hive is searching the
tickets and what Hive is reading??

Thanks so much for your help.

Best regards,

This message, and any attachments, is for the intended recipient(s) only, may contain information
that is privileged, confidential and/or proprietary and subject to important terms and conditions
available at If you are not the intended recipient,
please delete this message.

View raw message