accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "roman.drapeko@baesystems.com" <roman.drap...@baesystems.com>
Subject RE: Accumulo and Kerberos
Date Tue, 26 Jan 2016 18:42:28 GMT
Hi Josh,

Yes, will do. Just in the meantime - I can see a different issue on slave nodes. If I try
to start in isolation (bin/start-here.sh) with or without doing kinit I always see the error
below.

2016-01-26 18:31:13,873 [start.Main] ERROR: Problem initializing the class loader
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.accumulo.start.Main.getClassLoader(Main.java:68)
        at org.apache.accumulo.start.Main.main(Main.java:52)
Caused by: org.apache.commons.vfs2.FileSystemException: Could not determine the type of file
"hdfs://<hostname>/platform/lib/.*.jar".
        at org.apache.commons.vfs2.provider.AbstractFileObject.attach(AbstractFileObject.java:1522)
        at org.apache.commons.vfs2.provider.AbstractFileObject.getType(AbstractFileObject.java:489)
        at org.apache.accumulo.start.classloader.vfs.AccumuloVFSClassLoader.resolve(AccumuloVFSClassLoader.java:143)
        at org.apache.accumulo.start.classloader.vfs.AccumuloVFSClassLoader.resolve(AccumuloVFSClassLoader.java:121)
        at org.apache.accumulo.start.classloader.vfs.AccumuloVFSClassLoader.getClassLoader(AccumuloVFSClassLoader.java:211)
        ... 6 more
Caused by: org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not
enabled.  Available:[TOKEN, KERBEROS]

I guess it might be different to what I observe on the master node. If I don't get ticket
explicitly, I get the error mentioned in the previous email. However if do (and it does not
matter for what user I have a ticket now - whether it's accumulo, hdfs or hive) - it works.
So I started to think, maybe the problem related to some action (for example to vfs as per
above) that tries to access HDFS before doing a proper authentication with Kerberos? Any ideas?

Also, if we go live with 1.7.0 - what approach would you recommend for renewing tickets? Does
it require stopping and starting the cluster?

Regards,
Roman



-----Original Message-----
From: Josh Elser [mailto:josh.elser@gmail.com]
Sent: 26 January 2016 18:10
To: user@accumulo.apache.org
Subject: Re: Accumulo and Kerberos

Hi Roman,

Accumulo services (TabletServer, Master, etc) all use a keytab to automatically obtain a ticket
from the KDC when they start up. You do not need to do anything with kinit when starting Accumulo.

One worry is ACCUMULO-4069[1] with all presently released versions (most notably 1.7.0 which
you are using). This is a bug in which services did not automatically renew their ticket.
We're working on a 1.7.1, but it's not out yet.

As for debugging your issue, take a look at the Kerberos section on debugging in the user
manual [2]. Take a very close look at the principal the service is using to obtain the ticket
and what the principal is for your keytab. A good sanity check is to make sure you can `kinit`
in the shell using the keytab and the correct principal (rule out the keytab being incorrect).

If you still get stuck, collect the output specifying -Dsun.security.krb5.debug=true in accumulo-env.sh
(per the instructions) and try enabling log4j DEBUG on org.apache.hadoop.security.UserGroupInformation.

- Josh

[1] https://issues.apache.org/jira/browse/ACCUMULO-4069
[2] http://accumulo.apache.org/1.7/accumulo_user_manual.html#_debugging

roman.drapeko@baesystems.com wrote:
> Hi there,
>
> Trying to setup Accumulo 1.7 on Kerberized cluster. Only interested in
> master/tablets to be kerberized (not end-users). Configured everything
> as per manual:
>
> 1)Created principals
>
> 2)Generated glob keytab
>
> 3)Modified accumulo-site.xml providing general.kerberos.keytab and
> general.kerberos.principal
>
> If I start as accumulo user I get: Caused by: GSSException: No valid
> credentials provided (Mechanism level: Failed to find any Kerberos
> tgt)
>
> However, if I give explicitly a token with kinit and keytab generated
> above in the shell - it works as expected. To my understanding
> Accumulo has to obtain tickets automatically? Or the idea is to write
> a cron job and apply kinit to every tablet server per day?
>
> Regards,
>
> Roman
>
> Please consider the environment before printing this email. This
> message should be regarded as confidential. If you have received this
> email in error please notify the sender and destroy it immediately.
> Statements of intent shall only become binding when confirmed in hard
> copy by an authorised signatory. The contents of this email may relate
> to dealings with other companies under the control of BAE Systems
> Applied Intelligence Limited, details of which can be found at
> http://www.baesystems.com/Businesses/index.htm.
Please consider the environment before printing this email. This message should be regarded
as confidential. If you have received this email in error please notify the sender and destroy
it immediately. Statements of intent shall only become binding when confirmed in hard copy
by an authorised signatory. The contents of this email may relate to dealings with other companies
under the control of BAE Systems Applied Intelligence Limited, details of which can be found
at http://www.baesystems.com/Businesses/index.htm.

Mime
View raw message