incubator-hcatalog-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Olga Natkovich <>
Subject RE: HCatalog HA deployment
Date Thu, 01 Sep 2011 18:58:17 GMT
Yes, that's one of the options we are considering.


-----Original Message-----
From: Devaraj Das [] 
Sent: Thursday, September 01, 2011 10:53 AM
Subject: Re: HCatalog HA deployment

Yeah, we could persist the security state in ZK too.

On Sep 1, 2011, at 10:46 AM, Olga Natkovich wrote:

> We have tried to setup VIP to guarantee that traffic goes to the same server in secure
mode but that did not work at least in the setup we had. So I believe we need to store tokens
in a shared store. Also, going forward we are likely to have more state. For instance we are
storing versioning information for Hcat-Hbase integration in Zookeper. So we need to design
this in a generic fashion.
> Olga
> -----Original Message-----
> From: Alan Gates [] 
> Sent: Thursday, September 01, 2011 10:40 AM
> To:
> Subject: Re: HCatalog HA deployment
> The Thrift server that HCatalog uses to service metastore requests is the other SPOF
in HCat.  In unsecure mode it does not track state and so starting two servers and putting
them behind a VIP should be fine.  However, to my knowledge no one has tested this setup and
if you are thinking of using it you should test it before you buy hardware, make installation
plans, etc.
> In secure mode some of the security tokens are kept on the Thrift servers, and thus you
cannot use a VIP server in a round robin fashion.  If you could set it up such that the same
client went to the same server for the duration of their kerberos tickets then I think it
would work (again, test this, as no one has as far as I know).  In this scenario fail over
would not be seamless for users who were talking to the failed server.  They would get authentication
errors when they failed over and would be forced to restart.
> Alan.
> On Aug 31, 2011, at 7:11 PM, Thomas Weise wrote:
>> Hello,
>> I'm looking into HA support for hcatalog. We are going to have HA support at the
metastore RDBMS level. Beyond that, which areas of the server need to be looked at to accomplish
failover running multiple hcatalog servers with a VIP?
>> What state outside the database is maintained by hcatalog that needs to be available
to other instances to accomplish a VIP based failover in secure deployment?
>> Thanks!
>> Thomas

View raw message