accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: kerberos auth, getDelegationToken
Date Sun, 07 Jun 2015 03:00:05 GMT
Are you running it via `mvn exec:java` by chance or netbeans?

http://mail-archives.apache.org/mod_mbox/accumulo-user/201411.mbox/%3C547A9071.1020704@gmail.com%3E

If that's just a background thread writing in Stats, it might just be a 
factor of how you're invoking the program and you can ignore it. I don't 
know enough about the inner-workings of GeoMesa to say one way or the other.

Xu (Simon) Chen wrote:
> Josh,
>
> Everything works well, except for one thing :-)
>
> I am running geomesa-quickstart program that ingest some data and then
> perform a simple query:
> https://github.com/geomesa/geomesa-quickstart
>
> For some reason, the following error is emitted consistently at the
> end of the execution, after outputting the correct result:
> 15/06/07 00:29:22 INFO zookeeper.ZooCache: Zookeeper error, will retry
> java.lang.InterruptedException
>          at java.lang.Object.wait(Native Method)
>          at java.lang.Object.wait(Object.java:503)
>          at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1342)
>          at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1036)
>          at org.apache.accumulo.fate.zookeeper.ZooCache$2.run(ZooCache.java:264)
>          at org.apache.accumulo.fate.zookeeper.ZooCache.retry(ZooCache.java:162)
>          at org.apache.accumulo.fate.zookeeper.ZooCache.get(ZooCache.java:289)
>          at org.apache.accumulo.fate.zookeeper.ZooCache.get(ZooCache.java:238)
>          at org.apache.accumulo.core.client.impl.Tables.getTableState(Tables.java:180)
>          at org.apache.accumulo.core.client.impl.ConnectorImpl.getTableId(ConnectorImpl.java:82)
>          at org.apache.accumulo.core.client.impl.ConnectorImpl.createBatchWriter(ConnectorImpl.java:128)
>          at org.locationtech.geomesa.core.stats.StatWriter$$anonfun$write$2.apply(StatWriter.scala:174)
>          at org.locationtech.geomesa.core.stats.StatWriter$$anonfun$write$2.apply(StatWriter.scala:156)
>          at scala.collection.immutable.Map$Map1.foreach(Map.scala:109)
>          at org.locationtech.geomesa.core.stats.StatWriter$.write(StatWriter.scala:156)
>          at org.locationtech.geomesa.core.stats.StatWriter$.drainQueue(StatWriter.scala:148)
>          at org.locationtech.geomesa.core.stats.StatWriter$.run(StatWriter.scala:116)
>          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>          at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
>          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>          at java.lang.Thread.run(Thread.java:745)
>
> This is more annoying than a real problem. I am new to both accumulo
> and geomesa, but I am curious what the problem might be.
>
> Thanks!
> -Simon
>
>
> On Sat, Jun 6, 2015 at 8:01 PM, Josh Elser<josh.elser@gmail.com>  wrote:
>> Great! Glad to hear it. Please let us know how it works out!
>>
>>
>> Xu (Simon) Chen wrote:
>>> Josh,
>>>
>>> You're right again.. Thanks!
>>>
>>> My ansible play actually pushed client.conf to all the server config
>>> directories, but didn't do anything for the clients, and that's my
>>> problem. Now kerberos is working great for me.
>>>
>>> Thanks again!
>>> -Simon
>>>
>>> On Sat, Jun 6, 2015 at 5:04 PM, Josh Elser<josh.elser@gmail.com>   wrote:
>>>> Simon,
>>>>
>>>> Did you create a client configuration file (~/.accumulo/config or
>>>> $ACCUMULO_CONF_DIR/client.conf)? You need to configure Accumulo clients
>>>> to
>>>> actually use SASL when you're trying to use Kerberos authentication. Your
>>>> server is expecting that, but I would venture a guess that your client
>>>> isn't.
>>>>
>>>> See
>>>> http://accumulo.apache.org/1.7/accumulo_user_manual.html#_configuration_3
>>>>
>>>>
>>>> Xu (Simon) Chen wrote:
>>>>> Josh,
>>>>>
>>>>> Thanks. It makes sense...
>>>>>
>>>>> I used a KerberosToken, but my program got stuck when running the
>>>>> following:
>>>>> new ZooKeeperInstance(instance, zookeepers).getConnector(user, krbToken)
>>>>>
>>>>> It looks like my client is stuck here:
>>>>>
>>>>>
>>>>> https://github.com/apache/accumulo/blob/master/core/src/main/java/org/apache/accumulo/core/client/impl/ConnectorImpl.java#L70
>>>>> failing in the receive part of
>>>>>
>>>>>
>>>>> org.apache.accumulo.core.client.impl.thrift.ClientService.Client.authenticate().
>>>>>
>>>>> On my tservers, I see the following:
>>>>>
>>>>> 2015-06-06 18:58:19,616 [server.TThreadPoolServer] ERROR: Error
>>>>> occurred during processing of message.
>>>>> java.lang.RuntimeException:
>>>>> org.apache.thrift.transport.TTransportException:
>>>>> java.net.SocketTimeoutException: Read timed out
>>>>>            at
>>>>>
>>>>> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
>>>>>            at
>>>>>
>>>>> org.apache.accumulo.core.rpc.UGIAssumingTransportFactory$1.run(UGIAssumingTransportFactory.java:51)
>>>>>            at
>>>>>
>>>>> org.apache.accumulo.core.rpc.UGIAssumingTransportFactory$1.run(UGIAssumingTransportFactory.java:48)
>>>>>            at java.security.AccessController.doPrivileged(Native Method)
>>>>>            at javax.security.auth.Subject.doAs(Subject.java:356)
>>>>>            at
>>>>>
>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1622)
>>>>>            at
>>>>>
>>>>> org.apache.accumulo.core.rpc.UGIAssumingTransportFactory.getTransport(UGIAssumingTransportFactory.java:48)
>>>>>            at
>>>>>
>>>>> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:208)
>>>>>            at
>>>>>
>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>>>            at
>>>>>
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>>>            at
>>>>>
>>>>> org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)
>>>>>            at java.lang.Thread.run(Thread.java:745)
>>>>> Caused by: org.apache.thrift.transport.TTransportException:
>>>>> java.net.SocketTimeoutException: Read timed out
>>>>>            at
>>>>>
>>>>> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129)
>>>>>            at
>>>>> org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
>>>>>            at
>>>>>
>>>>> org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:182)
>>>>>            at
>>>>>
>>>>> org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125)
>>>>>            at
>>>>> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:253)
>>>>>            at
>>>>>
>>>>> org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
>>>>>            at
>>>>>
>>>>> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
>>>>>            ... 11 more
>>>>> Caused by: java.net.SocketTimeoutException: Read timed out
>>>>>            at java.net.SocketInputStream.socketRead0(Native Method)
>>>>>            at java.net.SocketInputStream.read(SocketInputStream.java:152)
>>>>>            at java.net.SocketInputStream.read(SocketInputStream.java:122)
>>>>>            at
>>>>> java.io.BufferedInputStream.read1(BufferedInputStream.java:273)
>>>>>            at
>>>>> java.io.BufferedInputStream.read(BufferedInputStream.java:334)
>>>>>            at
>>>>>
>>>>> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
>>>>>            ... 17 more
>>>>>
>>>>> Any ideas why?
>>>>>
>>>>> Thanks.
>>>>> -Simon
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Sat, Jun 6, 2015 at 2:19 PM, Josh Elser<josh.elser@gmail.com>
>>>>> wrote:
>>>>>> Make sure you read the JavaDoc on DelegationToken:
>>>>>>
>>>>>> <snip>
>>>>>> Obtain a delegation token by calling {@link
>>>>>>
>>>>>>
>>>>>> SecurityOperations#getDelegationToken(org.apache.accumulo.core.client.admin.DelegationTokenConfig)}
>>>>>> </snip>
>>>>>>
>>>>>> You cannot create a usable DelegationToken as the client itself.
>>>>>>
>>>>>> Anyways, DelegationTokens are only relevant in cases where the client
>>>>>> Kerberos credentials are unavailable. The most common case is running
>>>>>> MapReduce jobs. If you are just interacting with Accumulo through
the
>>>>>> Java
>>>>>> API, the KerberosToken is all you need to use.
>>>>>>
>>>>>> The user-manual likely just needs to be updated. I believe the
>>>>>> DelegationTokenConfig was added after I wrote the initial
>>>>>> documentation.
>>>>>>
>>>>>>
>>>>>> Xu (Simon) Chen wrote:
>>>>>>> Hi folks,
>>>>>>>
>>>>>>> The latest kerberos doc seems to indicate that getDelegationToken
can
>>>>>>> be
>>>>>>> called without any parameters:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> https://github.com/apache/accumulo/blob/1.7/docs/src/main/asciidoc/chapters/kerberos.txt#L410
>>>>>>>
>>>>>>> Yet the source code indicates a DelegationTokenConfig object
must be
>>>>>>> passed in:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> https://github.com/apache/accumulo/blob/1.7/core/src/main/java/org/apache/accumulo/core/client/admin/SecurityOperations.java#L359
>>>>>>>
>>>>>>> Any ideas on how I should construct the DelegationTokenConfig
object?
>>>>>>>
>>>>>>> For context, I've been trying to get geomesa to work on my accumulo
>>>>>>> 1.7
>>>>>>> with kerberos turned on. Right now, the code is somewhat tied
to
>>>>>>> password auth:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> https://github.com/locationtech/geomesa/blob/rc7_a1.7_h2.5/geomesa-core/src/main/scala/org/locationtech/geomesa/core/data/AccumuloDataStoreFactory.scala#L177
>>>>>>> My thought is that I should get a KerberosToken first, and then
try
>>>>>>> generate a DelegationToken, which is passed back for later
>>>>>>> interactions
>>>>>>> between geomesa and accumulo.
>>>>>>>
>>>>>>> Thanks.
>>>>>>> -Simon

Mime
View raw message