hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Elazar Leibovich <elaz...@gmail.com>
Subject "Server not found in Kerberos database" for MiniKDC server
Date Wed, 15 Jan 2014 20:57:56 GMT
Hi,

For educational purposes, I'm trying to set a minimal working secure Hadoop
cluster on my machine.

What I basically did is:

Add example.com to /etc/hosts

Set a minkdc server. It'll generate krb5.conf and keytab. Generates some
users - {nn,dn,hdfs}@EXAMPLE.COM

Refer Java to krb5.conf with HADOOP_OPTS, as well as a required workaround
for Mac OS X java:

❯ ~/hadoopconf env HADOOP_OPTS
hadoop-env.sh HADOOP_OPTS = -Djava.awt.headless=true
-Djava.security.krb5.conf=/Users/eleibovi/dev/securehadoop/hadoop_rpc_walktrhough/krb5.conf
-Djava.net.preferIPv4Stack=true


Set the proper hadoop configuration using the keytab and the hadoop users:

❯ ~/hadoopconf get --local
hdfs-site.xml dfs.datanode.address            = example.com:1004
core-site.xml fs.defaultFS                    = hdfs://example.com
hdfs-site.xml dfs.namenode.keytab.file        =
/Users/eleibovi/dev/securehadoop/hadoop_rpc_walktrhough/keytab
hdfs-site.xml dfs.datanode.hostname           = example.com
hdfs-site.xml dfs.datanode.kerberos.principal = dn/EXAMPLE.COM@EXAMPLE.COM
hdfs-site.xml dfs.datanode.data.dir           =
/tmp/hadoop-eleibovi/dfs/data
hdfs-site.xml dfs.datanode.keytab.file        =
/Users/eleibovi/dev/securehadoop/hadoop_rpc_walktrhough/keytab
hdfs-site.xml dfs.namenode.kerberos.principal = nn/EXAMPLE.COM@EXAMPLE.COM
core-site.xml hadoop.security.authorization   = true
core-site.xml hadoop.security.authentication  = kerberos
hdfs-site.xml dfs.datanode.dns.interface      = lo0
hdfs-site.xml dfs.datanode.http.address       = example.com:1006

Start the namenode service.
$ ./bin/hdfs
...
14/01/15 19:22:43 INFO ipc.Server: IPC Server listener on 8020: starting
14/01/15 19:22:43 INFO namenode.NameNode: NameNode RPC up at: localhost/
127.0.0.1:8020
14/01/15 19:22:43 INFO namenode.FSNamesystem: Starting services required
for active state

Finally use the following short Java program to contact the namenode:

System.setProperty("java.security.krb5.conf", cwd + "/krb5.conf");
UserGroupInformation.setConfiguration(conf);
        UserGroupInformation ugi = UserGroupInformation.
                loginUserFromKeytabAndReturnUGI("hdfs/EXAMPLE.COM", cwd +
"/keytab");
 ugi.doAs(new PrivilegedExceptionAction<Object>() {
            @Override
            public Object run() throws Exception {
                final FileSystem fs = FileSystem.get(conf);
                fs.getFileStatus(new Path("/"));
         }
}

The exception I got is:

Exception in thread "main" java.io.IOException: Failed on local exception:
java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed
[Caused by GSSException: No valid credentials provided (Mechanism level:
Server not found in Kerberos database (7) - Server not found in Kerberos
database)]; Host Details : local host is: "tlv-mpbxb/127.0.0.1";
destination host is: "example.com":8020;

I'll be glad to any help with debugging the problem.

Thanks,

I attach a full log with Kerberos debug turned on:

args: [-conf,
/Users/eleibovi/dev/securehadoop/hadoop-2.1.0-beta/etc/hadoop/core-site.xml,
-conf,
/Users/eleibovi/dev/securehadoop/hadoop-2.1.0-beta/etc/hadoop/hdfs-site.xml]
2014-01-15 19:29:46 DEBUG MutableMetricsFactory:42 - field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
with annotation
@org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, about=,
value=[Rate of successful kerberos logins and latency (milliseconds)],
always=false, type=DEFAULT, sampleName=Ops)
2014-01-15 19:29:46 DEBUG MutableMetricsFactory:42 - field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
with annotation
@org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, about=,
value=[Rate of failed kerberos logins and latency (milliseconds)],
always=false, type=DEFAULT, sampleName=Ops)
2014-01-15 19:29:46 DEBUG MetricsSystemImpl:220 - UgiMetrics, User and
group related metrics
Java config name:
/Users/eleibovi/dev/securehadoop/hadoop_rpc_walktrhough/krb5.conf
Loaded from Java config
2014-01-15 19:29:46 DEBUG Groups:180 -  Creating new Groups object
2014-01-15 19:29:46 DEBUG NativeCodeLoader:46 - Trying to load the
custom-built native-hadoop library...
2014-01-15 19:29:46 DEBUG NativeCodeLoader:55 - Failed to load
native-hadoop with error: java.lang.UnsatisfiedLinkError: no hadoop in
java.library.path
2014-01-15 19:29:46 DEBUG NativeCodeLoader:56 -
java.library.path=/Users/eleibovi/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.
2014-01-15 19:29:46 WARN  NativeCodeLoader:62 - Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
2014-01-15 19:29:46 DEBUG JniBasedUnixGroupsMappingWithFallback:40 -
Falling back to shell based
2014-01-15 19:29:46 DEBUG JniBasedUnixGroupsMappingWithFallback:44 - Group
mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
2014-01-15 19:29:46 DEBUG Groups:66 - Group mapping
impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;
cacheTimeout=300000
Java config name:
/Users/eleibovi/dev/securehadoop/hadoop_rpc_walktrhough/krb5.conf
Loaded from Java config
>>> KdcAccessibility: reset
>>> KdcAccessibility: reset
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): nn
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTab: load() entry length: 53; type: 3
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): nn
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTab: load() entry length: 69; type: 16
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): nn
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTab: load() entry length: 61; type: 17
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): nn
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTab: load() entry length: 61; type: 23
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): dn
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTab: load() entry length: 53; type: 3
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): dn
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTab: load() entry length: 69; type: 16
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): dn
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTab: load() entry length: 61; type: 17
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): dn
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTab: load() entry length: 61; type: 23
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): eleibovi
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTab: load() entry length: 59; type: 3
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): eleibovi
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTab: load() entry length: 75; type: 16
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): eleibovi
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTab: load() entry length: 67; type: 17
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): eleibovi
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTab: load() entry length: 67; type: 23
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): hdfs
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTab: load() entry length: 55; type: 3
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): hdfs
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTab: load() entry length: 71; type: 16
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): hdfs
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTab: load() entry length: 63; type: 17
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): hdfs
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTab: load() entry length: 63; type: 23
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): hdfs
>>> KeyTab: load() entry length: 42; type: 3
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): hdfs
>>> KeyTab: load() entry length: 58; type: 16
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): hdfs
>>> KeyTab: load() entry length: 50; type: 17
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): hdfs
>>> KeyTab: load() entry length: 50; type: 23
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): root
>>> KeyTab: load() entry length: 42; type: 3
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): root
>>> KeyTab: load() entry length: 58; type: 16
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): root
>>> KeyTab: load() entry length: 50; type: 17
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): root
>>> KeyTab: load() entry length: 50; type: 23
Added key: 23version: 0
Added key: 17version: 0
Added key: 16version: 0
Added key: 3version: 0
Ordering keys wrt default_tkt_enctypes list
Using builtin default etypes for default_tkt_enctypes
default etypes for default_tkt_enctypes: 18 17 16 23 1 3.
Added key: 23version: 0
Added key: 17version: 0
Added key: 16version: 0
Added key: 3version: 0
Ordering keys wrt default_tkt_enctypes list
Using builtin default etypes for default_tkt_enctypes
default etypes for default_tkt_enctypes: 18 17 16 23 1 3.
Using builtin default etypes for default_tkt_enctypes
default etypes for default_tkt_enctypes: 18 17 16 23 1 3.
>>> KrbAsReq creating message
>>> KrbKdcReq send: kdc=localhost TCP:50064, timeout=30000, number of
retries =3, #bytes=158
>>> KDCCommunication: kdc=localhost TCP:50064, timeout=30000,Attempt =1,
#bytes=158
>>>DEBUG: TCPClient reading 529 bytes
>>> KrbKdcReq send: #bytes read=529
>>> KdcAccessibility: remove localhost:50064
Added key: 23version: 0
Added key: 17version: 0
Added key: 16version: 0
Added key: 3version: 0
Ordering keys wrt default_tkt_enctypes list
Using builtin default etypes for default_tkt_enctypes
default etypes for default_tkt_enctypes: 18 17 16 23 1 3.
>>> EType: sun.security.krb5.internal.crypto.Aes128CtsHmacSha1EType
>>> KrbAsRep cons in KrbAsReq.getReply hdfs/EXAMPLE.COM
2014-01-15 19:29:47 DEBUG UserGroupInformation:176 - hadoop login
Added key: 23version: 0
Added key: 17version: 0
Added key: 16version: 0
Added key: 3version: 0
Ordering keys wrt default_tkt_enctypes list
Using builtin default etypes for default_tkt_enctypes
default etypes for default_tkt_enctypes: 18 17 16 23 1 3.
2014-01-15 19:29:47 DEBUG UserGroupInformation:125 - hadoop login commit
2014-01-15 19:29:47 DEBUG UserGroupInformation:139 - using kerberos
user:hdfs/EXAMPLE.COM@EXAMPLE.COM
2014-01-15 19:29:47 DEBUG UserGroupInformation:1499 - PrivilegedAction
as:hdfs/EXAMPLE.COM@EXAMPLE.COM (auth:KERBEROS)
from:com.github.elazar.hadoop.examples.HadoopBasicUsage.run(HadoopBasicUsage.java:34)
>>KERBEROS
2014-01-15 19:29:47 DEBUG BlockReaderLocal:326 -
dfs.client.use.legacy.blockreader.local = false
2014-01-15 19:29:47 DEBUG BlockReaderLocal:329 -
dfs.client.read.shortcircuit = false
2014-01-15 19:29:47 DEBUG BlockReaderLocal:332 -
dfs.client.domain.socket.data.traffic = false
2014-01-15 19:29:47 DEBUG BlockReaderLocal:335 - dfs.domain.socket.path =
2014-01-15 19:29:47 DEBUG MetricsSystemImpl:220 - StartupProgress, NameNode
startup progress
2014-01-15 19:29:47 DEBUG RetryUtils:74 - multipleLinearRandomRetry = null
2014-01-15 19:29:47 DEBUG Server:220 - rpcKind=RPC_PROTOCOL_BUFFER,
rpcRequestWrapperClass=class
org.apache.hadoop.ipc.ProtobufRpcEngine$RpcRequestWrapper,
rpcInvoker=org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker@379b0d9c
2014-01-15 19:29:47 DEBUG BlockReaderLocal:63 - Both short-circuit local
reads and UNIX domain socket are disabled.
2014-01-15 19:29:47 DEBUG Shell:237 - Failed to detect a valid hadoop home
directory
java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:219)
at org.apache.hadoop.util.Shell.<clinit>(Shell.java:244)
 at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:76)
at
org.apache.hadoop.conf.Configuration.getTrimmedStrings(Configuration.java:1539)
 at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:492)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:445)
 at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:136)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2429)
 at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:88)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2463)
 at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2445)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:363)
 at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:165)
at
com.github.elazar.hadoop.examples.HadoopBasicUsage$1.run(HadoopBasicUsage.java:38)
 at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
 at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1477)
at
com.github.elazar.hadoop.examples.HadoopBasicUsage.run(HadoopBasicUsage.java:34)
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
 at
com.github.elazar.hadoop.examples.HadoopBasicUsage.main(HadoopBasicUsage.java:18)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
2014-01-15 19:29:47 DEBUG Shell:316 - setsid is not available on this
machine. So not using it.
2014-01-15 19:29:47 DEBUG Shell:320 - setsid exited with exit code 0
2014-01-15 19:29:47 DEBUG Client:371 - The ping interval is 60000 ms.
2014-01-15 19:29:47 DEBUG Client:636 - Connecting to
example.com/127.0.0.1:8020
2014-01-15 19:29:47 DEBUG UserGroupInformation:1499 - PrivilegedAction
as:hdfs/EXAMPLE.COM@EXAMPLE.COM (auth:KERBEROS)
from:org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:654)
2014-01-15 19:29:47 DEBUG SaslRpcClient:438 - Sending sasl message state:
NEGOTIATE

2014-01-15 19:29:47 DEBUG SaslRpcClient:370 - Received SASL message state:
NEGOTIATE
auths {
  method: "TOKEN"
  mechanism: "DIGEST-MD5"
  protocol: ""
  serverId: "default"
  challenge:
"realm=\"default\",nonce=\"Evyt9cWyZFDUbCtQYNcgF5FY7rsBqxVCgtggY48n\",qop=\"auth\",charset=utf-8,algorithm=md5-sess"
}
auths {
  method: "KERBEROS"
  mechanism: "GSSAPI"
  protocol: "nn"
  serverId: "EXAMPLE.COM"
}

2014-01-15 19:29:47 DEBUG SaslRpcClient:259 - Get token info
proto:interface org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolPB
info:@org.apache.hadoop.security.token.TokenInfo(value=class
org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenSelector)
2014-01-15 19:29:47 DEBUG SaslRpcClient:287 - Get kerberos info
proto:interface org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolPB
info:@org.apache.hadoop.security.KerberosInfo(clientPrincipal=,
serverPrincipal=dfs.namenode.kerberos.principal)
2014-01-15 19:29:47 DEBUG SaslRpcClient:231 - RPC Server's Kerberos
principal name for
protocol=org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolPB is nn/
EXAMPLE.COM@EXAMPLE.COM
2014-01-15 19:29:47 DEBUG SaslRpcClient:242 - Creating SASL
GSSAPI(KERBEROS)  client to authenticate to service at EXAMPLE.COM
2014-01-15 19:29:47 DEBUG SaslRpcClient:172 - Use KERBEROS authentication
for protocol ClientNamenodeProtocolPB
Found ticket for hdfs/EXAMPLE.COM@EXAMPLE.COM to go to krbtgt/
EXAMPLE.COM@EXAMPLE.COM expiring on Thu Jan 16 19:29:46 IST 2014
Entered Krb5Context.initSecContext with state=STATE_NEW
Found ticket for hdfs/EXAMPLE.COM@EXAMPLE.COM to go to krbtgt/
EXAMPLE.COM@EXAMPLE.COM expiring on Thu Jan 16 19:29:46 IST 2014
Service ticket not found in the subject
>>> Credentials acquireServiceCreds: same realm
Using builtin default etypes for default_tgs_enctypes
default etypes for default_tgs_enctypes: 18 17 16 23 1 3.
>>> CksumType: sun.security.krb5.internal.crypto.RsaMd5CksumType
>>> EType: sun.security.krb5.internal.crypto.Aes128CtsHmacSha1EType
>>> KrbKdcReq send: kdc=localhost TCP:50064, timeout=30000, number of
retries =3, #bytes=583
>>> KDCCommunication: kdc=localhost TCP:50064, timeout=30000,Attempt =1,
#bytes=583
>>>DEBUG: TCPClient reading 135 bytes
>>> KrbKdcReq send: #bytes read=135
>>> KdcAccessibility: remove localhost:50064
>>> KDCRep: init() encoding tag is 126 req type is 13
>>>KRBError:
 sTime is Wed Jan 15 19:29:47 IST 2014 1389806987000
 suSec is 0
 error code is 7
 error Message is Server not found in Kerberos database
 realm is EXAMPLE.COM
 sname is krbtgt/EXAMPLE.COM
 msgType is 30
2014-01-15 19:29:47 ERROR UserGroupInformation:1480 -
PriviledgedActionException as:hdfs/EXAMPLE.COM@EXAMPLE.COM (auth:KERBEROS)
cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by
GSSException: No valid credentials provided (Mechanism level: Server not
found in Kerberos database (7) - Server not found in Kerberos database)]
2014-01-15 19:29:47 DEBUG UserGroupInformation:1499 - PrivilegedAction
as:hdfs/EXAMPLE.COM@EXAMPLE.COM (auth:KERBEROS)
from:org.apache.hadoop.ipc.Client$Connection.handleSaslConnectionFailure(Client.java:583)
>>>KinitOptions cache name is /Users/eleibovi/krb5cc_eleibovi
>> Acquire default native Credentials
>>> Obtained TGT from LSA: Credentials:
*...Here it looks like it tries to get the machines kerberos tickets...*
client=eleibovi@[snipped computer Kerberos server from /etc/krb5.conf]
server=krbtgt/[snipped computer Kerberos server from /etc/krb5.conf]
authTime=20140109035156Z
startTime=20140109115308Z
endTime=20140109215308Z
renewTill=20140116035156Z
flags: FORWARDABLE;RENEWABLE;INITIAL;PRE-AUTHENT
EType (int): 18
Using builtin default etypes for default_tgs_enctypes
default etypes for default_tgs_enctypes: 18 17 16 23 1 3.
>>> CksumType: sun.security.krb5.internal.crypto.RsaMd5CksumType
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
getKDCFromDNS using UDP
getKDCFromDNS using TCP
2014-01-15 19:30:18 DEBUG UserGroupInformation:176 - hadoop login
2014-01-15 19:30:18 DEBUG UserGroupInformation:125 - hadoop login commit
2014-01-15 19:30:18 DEBUG UserGroupInformation:139 - using kerberos
user:null
2014-01-15 19:30:18 DEBUG UserGroupInformation:155 - using local
user:UnixPrincipal: eleibovi
2014-01-15 19:30:18 DEBUG UserGroupInformation:696 - UGI loginUser:eleibovi
(auth:KERBEROS)
2014-01-15 19:30:18 WARN  Client:615 - Exception encountered while
connecting to the server : javax.security.sasl.SaslException: GSS initiate
failed [Caused by GSSException: No valid credentials provided (Mechanism
level: Server not found in Kerberos database (7) - Server not found in
Kerberos database)]
2014-01-15 19:30:18 ERROR UserGroupInformation:1480 -
PriviledgedActionException as:hdfs/EXAMPLE.COM@EXAMPLE.COM (auth:KERBEROS)
cause:java.io.IOException: javax.security.sasl.SaslException: GSS initiate
failed [Caused by GSSException: No valid credentials provided (Mechanism
level: Server not found in Kerberos database (7) - Server not found in
Kerberos database)]
2014-01-15 19:30:18 DEBUG Client:1099 - closing ipc connection to
example.com/127.0.0.1:8020: javax.security.sasl.SaslException: GSS initiate
failed [Caused by GSSException: No valid credentials provided (Mechanism
level: Server not found in Kerberos database (7) - Server not found in
Kerberos database)]
java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed
[Caused by GSSException: No valid credentials provided (Mechanism level:
Server not found in Kerberos database (7) - Server not found in Kerberos
database)]
at org.apache.hadoop.ipc.Client$Connection$1.run(Client.java:620)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1477)
at
org.apache.hadoop.ipc.Client$Connection.handleSaslConnectionFailure(Client.java:583)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:667)
at org.apache.hadoop.ipc.Client$Connection.access$2600(Client.java:314)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1399)
at org.apache.hadoop.ipc.Client.call(Client.java:1318)
at org.apache.hadoop.ipc.Client.call(Client.java:1300)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:188)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:651)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1636)
at
org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1117)
at
org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1113)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:78)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1113)
at
com.github.elazar.hadoop.examples.HadoopBasicUsage$1.run(HadoopBasicUsage.java:40)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1477)
at
com.github.elazar.hadoop.examples.HadoopBasicUsage.run(HadoopBasicUsage.java:34)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at
com.github.elazar.hadoop.examples.HadoopBasicUsage.main(HadoopBasicUsage.java:18)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
Caused by: javax.security.sasl.SaslException: GSS initiate failed [Caused
by GSSException: No valid credentials provided (Mechanism level: Server not
found in Kerberos database (7) - Server not found in Kerberos database)]
at
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
at
org.apache.hadoop.security.SaslRpcClient.saslConnect(SaslRpcClient.java:394)
at
org.apache.hadoop.ipc.Client$Connection.setupSaslConnection(Client.java:494)
at org.apache.hadoop.ipc.Client$Connection.access$1700(Client.java:314)
at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:659)
at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:655)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1477)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:654)
... 32 more
Caused by: GSSException: No valid credentials provided (Mechanism level:
Server not found in Kerberos database (7) - Server not found in Kerberos
database)
at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
... 41 more
Caused by: KrbException: Server not found in Kerberos database (7) - Server
not found in Kerberos database
at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:73)
at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:192)
at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:203)
at
sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:311)
at
sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:115)
at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:449)
at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:641)
... 44 more
Caused by: KrbException: Identifier doesn't match expected value (906)
at sun.security.krb5.internal.KDCRep.init(KDCRep.java:143)
at sun.security.krb5.internal.TGSRep.init(TGSRep.java:66)
at sun.security.krb5.internal.TGSRep.<init>(TGSRep.java:61)
at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:55)
... 50 more
2014-01-15 19:30:18 DEBUG Client:1107 - IPC Client (23781497) connection to
example.com/127.0.0.1:8020 from hdfs/EXAMPLE.COM@EXAMPLE.COM: closed

Mime
View raw message