accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <>
Subject Re: ClientConfiguration using Kerberos & MapReduce
Date Thu, 18 May 2017 21:20:06 GMT
James Srinivasan wrote:
>> [snip]
>> ConfiguratorBase has no other overrides for setZookeeperInstance, so I
>> don't see how this would ever work with Kerberos. It is marked as
>> deprecated, which points me to AccumuloInputFormat, but I'm a little
>> confused as to how this API relates to ConfiguratorBase.
> So I'm now comparing (for example):
> ConfiguratorBase.setConnectorInfo(classOf[AccumuloInputFormat], conf,
> username, password)
> (old API)
> with
> AbstractInputFormat.setConnectorInfo(new Job(conf), username, password)
> (new API)
> The difference is that the old API operates directly on the
> Configuration and updates it in-place, whereas my way of calling the
> new API seems to create a copy of the Configuration and leave the
> original untouched. This leaves me with a problem of having to merge
> the old and new Configurations - surely there must be a better way?
> Thanks,
> James

(whoops, forgot to respond to your first message)

You've definitely stumbled onto a very painful corner of our public API. 
We got screwed (essentially) by introducing a bunch of code that wasn't 
really meant to be public API (stable) while trying to consolidate our 
implementation between the Hadoop mapred and mapreduce API calls and the 
InputFormat/OutputFormat for each. Anyways!

To your original question: you'd want to look at the method,

`AccumuloInputFormat.setConnectorInfo(Job, String, AuthenticationToken)` 
(the implementation is actually on AbstractInputFormat if you're curious..)

You would construct a KerberosToken via normal methods (Instance + 
ClientConfiguration) and pass that to this method. When you do this, the 
implementation automatically fetches delegation tokens for you (tl;dr on 
delegation tokens: short-lived password sufficient to identify you that 
prevents us from having to distribute your Kerberos credentials across 
the cluster).

Fair-warning: you'll need to make sure you grant the permission to your 
user to obtain delegation tokens (System.OBTAIN_DELEGATION_TOKEN) 
otherwise you'll get an permission error from the Master when the 
Input/OutputFormat asks for one on your behalf.

View raw message