accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xu (Simon) Chen" <xche...@gmail.com>
Subject Re: same token couldn't authenticate twice?
Date Fri, 12 Jun 2015 00:32:57 GMT
Jim,

The commit you mentioned is not in the branch you created for me
(rc7_a1.7_h2.5)...

-Simon

On Thu, Jun 11, 2015 at 8:09 PM, James Hughes <jnh5y@virginia.edu> wrote:
> Simon,
>
> Hmm... it may be worth reverting the changes to the GeoMesaSpark source from
> this commit:
> https://github.com/locationtech/geomesa/commit/1d0178e399ce5f66f0bcd94b071bf0634fe31e8d.
>
> That'd give a chance to make sure the problem is with the AccumuloInput
> rather than something GeoMesa is doing with the input format.  As note,
> GeoMesa has some code to help gather splits similar to what Eugene proposed
> recently.  I don't know if that made it into 1.7.0 or not.
>
> Cheers,
>
> Jim
>
> On Thu, Jun 11, 2015 at 7:51 PM, Xu (Simon) Chen <xchenum@gmail.com> wrote:
>>
>> Thanks Josh...
>>
>> I tested this in scala REPL, and called DataStoreFinder.getDataStore()
>> multiple times, each time it seems to be reusing the same
>> KerberosToken object, and it works fine each time.
>>
>> So my problem only happens when the token is used in accumulo's mapred
>> package. Weird..
>>
>> -Simon
>>
>>
>> On Thu, Jun 11, 2015 at 5:29 PM, Josh Elser <josh.elser@gmail.com> wrote:
>> > Simon,
>> >
>> > Can you reproduce this in plain-jane Java code? I don't know enough
>> > about
>> > spark/scala, much less what Geomesa is actually do, to know what the
>> > issue
>> > is.
>> >
>> > Also, which token are you referring to: A KerberosToken or a
>> > DelegationToken? Either of them should be usable as many times as you'd
>> > like
>> > (given the underlying credentials are still available for KT or the DT
>> > token
>> > hasn't yet expired).
>> >
>> >
>> > Xu (Simon) Chen wrote:
>> >>
>> >> Folks,
>> >>
>> >> I am working on geomesa+accumulo+spark integration. For some reason, I
>> >> found that the same token cannot be used to authenticate twice.
>> >>
>> >> The workflow is that geomesa would try to create a hadoop rdd, during
>> >> which it tries to create an AccumuloDataStore:
>> >>
>> >>
>> >> https://github.com/locationtech/geomesa/blob/master/geomesa-compute/src/main/scala/org/locationtech/geomesa/compute/spark/GeoMesaSpark.scala#L81
>> >>
>> >> During this process, a ZooKeeperInstance is created:
>> >>
>> >>
>> >> https://github.com/locationtech/geomesa/blob/rc7_a1.7_h2.5/geomesa-core/src/main/scala/org/locationtech/geomesa/core/data/AccumuloDataStoreFactory.scala#L177
>> >> I modified geomesa such that it would use kerberos to authenticate
>> >> here. This step works fine.
>> >>
>> >> But next, geomesa calls ConfigurationBase.setConnectorInfo:
>> >>
>> >>
>> >> https://github.com/locationtech/geomesa/blob/rc7_a1.7_h2.5/geomesa-compute/src/main/scala/org/locationtech/geomesa/compute/spark/GeoMesaSpark.scala#L69
>> >>
>> >> This is using the same token and the same zookeeper URI, for some
>> >> reason it is stuck on spark-shell, and the following is outputted on
>> >> tserver side:
>> >>
>> >> 2015-06-06 18:58:19,616 [server.TThreadPoolServer] ERROR: Error
>> >> occurred during processing of message.
>> >> java.lang.RuntimeException:
>> >> org.apache.thrift.transport.TTransportException:
>> >> java.net.SocketTimeoutException: Read timed out
>> >>          at
>> >>
>> >> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
>> >>          at
>> >>
>> >> org.apache.accumulo.core.rpc.UGIAssumingTransportFactory$1.run(UGIAssumingTransportFactory.java:51)
>> >>          at
>> >>
>> >> org.apache.accumulo.core.rpc.UGIAssumingTransportFactory$1.run(UGIAssumingTransportFactory.java:48)
>> >>          at java.security.AccessController.doPrivileged(Native Method)
>> >>          at javax.security.auth.Subject.doAs(Subject.java:356)
>> >>          at
>> >>
>> >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1622)
>> >>          at
>> >>
>> >> org.apache.accumulo.core.rpc.UGIAssumingTransportFactory.getTransport(UGIAssumingTransportFactory.java:48)
>> >>          at
>> >>
>> >> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:208)
>> >>          at
>> >>
>> >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> >>          at
>> >>
>> >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> >>          at
>> >>
>> >> org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)
>> >>          at java.lang.Thread.run(Thread.java:745)
>> >> Caused by: org.apache.thrift.transport.TTransportException:
>> >> java.net.SocketTimeoutException: Read timed out
>> >>          at
>> >>
>> >> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129)
>> >>          at
>> >> org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
>> >>          at
>> >>
>> >> org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:182)
>> >>          at
>> >>
>> >> org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125)
>> >>          at
>> >>
>> >> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:253)
>> >>          at
>> >>
>> >> org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
>> >>          at
>> >>
>> >> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
>> >>          ... 11 more
>> >> Caused by: java.net.SocketTimeoutException: Read timed out
>> >>          at java.net.SocketInputStream.socketRead0(Native Method)
>> >>          at java.net.SocketInputStream.read(SocketInputStream.java:152)
>> >>          at java.net.SocketInputStream.read(SocketInputStream.java:122)
>> >>          at
>> >> java.io.BufferedInputStream.read1(BufferedInputStream.java:273)
>> >>          at
>> >> java.io.BufferedInputStream.read(BufferedInputStream.java:334)
>> >>          at
>> >>
>> >> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
>> >>          ... 17 more
>> >>
>> >> Any idea why?
>> >>
>> >> Thanks.
>> >> -Simon
>
>

Mime
View raw message