hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Parth Brahmbhatt <pbrahmbh...@hortonworks.com>
Subject Re: getting delegation token for hbase
Date Fri, 15 Aug 2014 20:51:10 GMT
Thanks for the reply. Storm topologies are by design suppose to run for ever, The only advantage
I can think of having a renewal mechanism is that instead of distributing the tokens to all
workers every "expiration millis” the master just renews it. When they eventually expire
(in HDFS’s case I think its 7 days) the storm master still has to get and push the new tokens
but the renewal reduces the push frequency thus reducing some work load from master. 

That is not to say Hbase should implement renewal but in the absence of it I hope the expiration
is relatively a larger number.


On Aug 15, 2014, at 1:35 PM, Gary Helmling <ghelmling@gmail.com> wrote:

>> I don’t think we need to support older versions of HBase. However there is
>> one thing that still bugs me. How does token renewal work here? Generally
>> in HDFS I have seen that you have to pass in the renewer user as an
>> argument when you obtain a token. Here as renew user is not passed I am
>> guessing it’s either some hardcoded Hbase value, or its derived from the
>> UGI.
> HBase doesn't really handle token renewal the way that, say, HDFS does.
> With HBase the token is simply valid for a fixed period.  In HDFS, the NN
> retains a map of all current tokens in memory and updates the expiration
> for a given token when it is renewed, but this is still subject to a max
> age, so that token still eventually expires.  In HBase, the authentication
> performed with the token is distributed (all regionservers can authenticate
> clients with the token), so keeping all tokens synchronized in memory on
> all nodes would be difficult.  I also don't think supporting renewal would
> add a great deal of value for this case.
> So for a truly long running process which could live beyond the token
> lifetime, you need to have your "delegator", which obtains the initial
> tokens, periodically obtain new tokens for the processes and make those
> available to the processes that need them.  The same will also be true for
> HDFS delegation tokens when your processes could run for longer than the
> token max age (maximum renewal time).

NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

View raw message