hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Weise (Commented) (JIRA)" <>
Subject [jira] [Commented] (HIVE-2467) HA Support for Metastore Server
Date Mon, 14 Nov 2011 16:53:51 GMT


Thomas Weise commented on HIVE-2467:

Yes, this was the intention. Also, the current ZK token store implementation could be translated
into a SQL implementation with relatively small effort (it does not depend on ZK watches for
invalidation). For our use, we plan to deal with the security limitation at L3 and use ZK
security when it becomes available. Longer term, assuming there will be use cases outside
HCatalog/Hive for long running processes that require token support in conjunction with HA
but without HA RDBMS already available, a ZK based token storage might be a more likely choice.
While plain HDFS is secure it does not support the access pattern needed here (fast random
reads and writes). 

> HA Support for Metastore Server 
> --------------------------------
>                 Key: HIVE-2467
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>          Components: Metastore, Security, Server Infrastructure
>    Affects Versions: 0.8.0, 0.9.0
>            Reporter: Thomas Weise
>            Assignee: Thomas Weise
>             Fix For: 0.9.0
>         Attachments: HIVE-2467.patch
> We require HA deployment for metastore server for HCatalog:
> * Multiple server instances run behind VIP
> * Database provides HA
> Metastore server instances will need to be able to share any state required for VIP outside
RDBMS. As of Hive 0.8 affected conversational state that needs to support VIP/HA setup is
limited to current delegation tokens. Is this correct?
> We are planning to use ZooKeeper to share current delegation tokens and master keys between
nodes of the VIP. ZK is already (optionally) used by Hive for concurrency control. Access
to ZK would be limited on the network level or in the future, when ZooKeeper supports security,
through Kerberos, similar to NN access.
> Currently Hive taps into Hadoop core security delegation token support through extension
> A solution could amend the Hive specific extension to support:
> * Pluggable delegation token and master key store (ZooKeeper as alternative for in-memory
> * Delegation token retrieval from token store when not found in memory (wrap/extend retrievePassword(...))
> * Cancellation of token in token store
> * Purging of expired tokens from token store

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message