accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: ClientConfiguration using Kerberos & MapReduce
Date Wed, 07 Jun 2017 20:31:55 GMT
On 6/7/17 3:54 PM, James Srinivasan wrote:
> [snip]
>>> Fortunately I found this:
>>>
>>> https://github.com/apache/hive/blob/master/accumulo-handler/src/java/org/apache/hadoop/hive/accumulo/mr/HiveAccumuloTableInputFormat.java
>>>
>>> Is it a good example of Accumulo + MapReduce that I can copy?
>> That one is definitely over-kill. There's a bit of reflection in there to
>> work around older versions of Accumulo. However, it should be an example of
>> something that does work with Kerberos authentication.
>> Also, take note that Hive uses the InputFormat regardless of the execution
>> engine (local, MapReduce, Tez, etc). There are some comments to that effect
>> in the code. You can likely simplify those methods/blocks as well :)
> 
> Think those are two things I'll need to handle at some point anyways.
> I think I'm setting all the AccumuloInputFormat statics correctly, and
> see the DelegationToken in my job's and context's credentials.
> However, my custom InputFormat's createRecordReader function needs to
> connect to Accumulo to get some config. Am I right in thinking I need
> to convert the Hadoop wrapped token (kind=ACCUMULO_AUTH_TOKEN) into an
> Accumulo DelegationToken to create my connector? If so, how do I do
> that?

Yes, you need to deserialize the AuthenticationToken from the 
InputSplit. You can look back into the AccumuloInputFormat 
implementation to see how this is done:

https://github.com/apache/accumulo/blob/f81a8ec7410e789d11941351d5899b8894c6a322/core/src/main/java/org/apache/accumulo/core/client/mapreduce/AbstractInputFormat.java#L515-L518

calls

https://github.com/apache/accumulo/blob/f81a8ec7410e789d11941351d5899b8894c6a322/core/src/main/java/org/apache/accumulo/core/client/mapreduce/AbstractInputFormat.java#L240-L243

calls

https://github.com/apache/accumulo/blob/f81a8ec7410e789d11941351d5899b8894c6a322/core/src/main/java/org/apache/accumulo/core/client/mapreduce/lib/impl/ConfiguratorBase.java#L485-L500

This pulls the "DelegationTokenStub" out of the InputFormat and creates 
a real Accumulo AuthenticationToken (which you can use with a Connector 
per-usual).

Mime
View raw message