hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tony Dean <Tony.D...@sas.com>
Subject hadoop kerberos security / unix kdc
Date Fri, 29 Jun 2012 20:50:23 GMT
First, I'd like to thank the community for the time and effort they put into sharing their

A few weeks back I was able to configure a secure hadoop/hbase cluster (MIT 1.6.1 Kerberos
on cluster) using a Windows Domain Controller/AD for the KDC.  I'm using hadoop 1.0.3 and
hbase 0.92.1-security distributions.

Now I am trying setup my own Unix KDC (MIT 1.9.1 Kerberos) against that same cluster.  I know
the cluster is configured correctly.  The only new piece to the puzzle is the Unix KDC.  The
problem occurs when I start the namenode.  It is actually able to login my namenode principal
into the KDC just fine.  I can see in the namenode main code that the HTTP Server as well
as the RPC server has been created successfully.  It's in the startTrashEmptier() method where
the error occurs.  It's like Hadoop is acting as a client and connecting back into itself
(hdfs service) when it receives a checksum error:

12/06/29 15:56:13 INFO security.UserGroupInformation: Login successful for user host/rdcesx10030.race.sas.com@OBSIDIAN.SAS.COM
using keytab file /etc/krb5.keytab
12/06/29 15:56:13 INFO ipc.Server: IPC Server Responder: starting
12/06/29 15:56:13 INFO ipc.Server: IPC Server listener on 8020: starting
Found key for host/rdcesx10030.race.sas.com@OBSIDIAN.SAS.COM(18)
Found key for host/rdcesx10030.race.sas.com@OBSIDIAN.SAS.COM(3)
Found key for host/rdcesx10030.race.sas.com@OBSIDIAN.SAS.COM(16)
Found key for host/rdcesx10030.race.sas.com@OBSIDIAN.SAS.COM(17)
Found key for host/rdcesx10030.race.sas.com@OBSIDIAN.SAS.COM(23)
Entered Krb5Context.acceptSecContext with state=STATE_NEW
>>> EType: sun.security.krb5.internal.crypto.ArcFourHmacEType
Checksum failed !
12/06/29 15:56:13 INFO ipc.Server: IPC Server listener on 8020: readAndProcess threw exception
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: Failure unspecified
at GSS-API level (Mechanism level: Checksum failed)]. Count of bytes read: 0
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: Failure unspecified
at GSS-API level (Mechanism level: Checksum failed)]
        at com.sun.security.sasl.gsskerb.GssKrb5Server.evaluateResponse(GssKrb5Server.java:159)
        at org.apache.hadoop.ipc.Server$Connection.saslReadAndProcess(Server.java:1007)
        at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1180)
        at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:537)
        at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:344)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)
Caused by: GSSException: Failure unspecified at GSS-API level (Mechanism level: Checksum failed)
        at sun.security.jgss.krb5.Krb5Context.acceptSecContext(Krb5Context.java:741)
        at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:323)
        at sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:267)
        at com.sun.security.sasl.gsskerb.GssKrb5Server.evaluateResponse(GssKrb5Server.java:137)
        ... 7 more
Caused by: KrbException: Checksum failed
        at sun.security.krb5.internal.crypto.ArcFourHmacEType.decrypt(ArcFourHmacEType.java:85)
        at sun.security.krb5.internal.crypto.ArcFourHmacEType.decrypt(ArcFourHmacEType.java:77)
        at sun.security.krb5.EncryptedData.decrypt(EncryptedData.java:168)
        at sun.security.krb5.KrbApReq.authenticate(KrbApReq.java:268)
        at sun.security.krb5.KrbApReq.<init>(KrbApReq.java:134)
        at sun.security.jgss.krb5.InitSecContextToken.<init>(InitSecContextToken.java:79)
        at sun.security.jgss.krb5.Krb5Context.acceptSecContext(Krb5Context.java:724)
        ... 10 more
Caused by: java.security.GeneralSecurityException: Checksum failed
        at sun.security.krb5.internal.crypto.dk.ArcFourCrypto.decrypt(ArcFourCrypto.java:388)
        at sun.security.krb5.internal.crypto.ArcFourHmac.decrypt(ArcFourHmac.java:74)
        at sun.security.krb5.internal.crypto.ArcFourHmacEType.decrypt(ArcFourHmacEType.java:83)
        ... 16 more

I think it has something to do with the keys in my keytab.  Although, I can kinit into the
KDC with all of the principals in my keytab so I don't know what the problem is.

I read something (not validated though) that there may be some incompatibility with Hadoop
security and MIT 1.9.1.

Any insight here would be greatly appreciated.


Tony Dean
SAS Institute Inc.
Senior Software Developer

  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message