hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-2467) HA Support for Metastore Server
Date Tue, 22 Nov 2011 02:28:42 GMT

    [ https://issues.apache.org/jira/browse/HIVE-2467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154829#comment-13154829
] 

jiraposter@reviews.apache.org commented on HIVE-2467:
-----------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2721/#review3417
-----------------------------------------------------------



trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java
<https://reviews.apache.org/r/2721/#comment7646>

    For consistency reasons, these should be moved to HiveConf. Also, the other properties
which are there in this class.



trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java
<https://reviews.apache.org/r/2721/#comment7647>

    Refer to it by Map.



trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java
<https://reviews.apache.org/r/2721/#comment7648>

    Instead of java.util.concurrent.ConcurrentHashMap, use ConcurrentHashMap instead.



trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java
<https://reviews.apache.org/r/2721/#comment7650>

    If TokenStore is meant to be Configurable, then have a private variable here to hold the
conf in setConf() and return that in getConf()



trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java
<https://reviews.apache.org/r/2721/#comment7651>

    use ArrayList, instead of fully qualified with package name



trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java
<https://reviews.apache.org/r/2721/#comment7652>

    Interface method needs javadoc.



trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java
<https://reviews.apache.org/r/2721/#comment7653>

    Interface method needs javadoc.



trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java
<https://reviews.apache.org/r/2721/#comment7654>

    Either it should return boolean or it should throw Exception. Doing both is confusing.
If you want to throw exception, then throw exception on failure and return void, else return
false in failure scenario. Token must be removed in all cases if method returns. 



trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java
<https://reviews.apache.org/r/2721/#comment7655>

    HashMap instead of java.util.HashMap



trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java
<https://reviews.apache.org/r/2721/#comment7656>

    Fully qualified classname.



trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java
<https://reviews.apache.org/r/2721/#comment7658>

    Should these come from hiveConf, so they are externally configurable?



trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java
<https://reviews.apache.org/r/2721/#comment7659>

    Contract of configurable is to return conf in getConf(), so you must store it in a private
variable and return it.



trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java
<https://reviews.apache.org/r/2721/#comment7660>

    One way to test HA is to start two metastore processes in two different threads configure
to use ZK token store and then do operation on first kill it and then do operation on second
one and then have it succeeds. If thats not straight forward, may be we can take that up in
separate ticket. 



trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java
<https://reviews.apache.org/r/2721/#comment7661>

    Get rid of commented code.


- Ashutosh


On 2011-11-17 00:57:32, Thomas wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2721/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2011-11-17 00:57:32)
bq.  
bq.  
bq.  Review request for Carl Steinbach, Ashutosh Chauhan and Devaraj Das.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  https://issues.apache.org/jira/browse/HIVE-2467
bq.  
bq.  
bq.  This addresses bug HIVE-2467.
bq.      https://issues.apache.org/jira/browse/HIVE-2467
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    trunk/shims/ivy.xml 1202918 
bq.    trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java
1202918 
bq.    trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java PRE-CREATION

bq.    trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java
PRE-CREATION 
bq.    trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java PRE-CREATION

bq.    trunk/shims/src/0.20S/java/org/apache/hadoop/security/token/delegation/HiveDelegationTokenSupport.java
PRE-CREATION 
bq.    trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java 1202918

bq.  
bq.  Diff: https://reviews.apache.org/r/2721/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  unit test added, ant clean package test  - passed
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Thomas
bq.  
bq.


                
> HA Support for Metastore Server 
> --------------------------------
>
>                 Key: HIVE-2467
>                 URL: https://issues.apache.org/jira/browse/HIVE-2467
>             Project: Hive
>          Issue Type: Improvement
>          Components: Metastore, Security, Server Infrastructure
>    Affects Versions: 0.8.0, 0.9.0
>            Reporter: Thomas Weise
>            Assignee: Thomas Weise
>             Fix For: 0.9.0
>
>         Attachments: HIVE-2467.2.patch, HIVE-2467.patch
>
>
> We require HA deployment for metastore server for HCatalog:
> * Multiple server instances run behind VIP
> * Database provides HA
> Metastore server instances will need to be able to share any state required for VIP outside
RDBMS. As of Hive 0.8 affected conversational state that needs to support VIP/HA setup is
limited to current delegation tokens. Is this correct?
> We are planning to use ZooKeeper to share current delegation tokens and master keys between
nodes of the VIP. ZK is already (optionally) used by Hive for concurrency control. Access
to ZK would be limited on the network level or in the future, when ZooKeeper supports security,
through Kerberos, similar to NN access.
> Currently Hive taps into Hadoop core security delegation token support through extension
of
> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager<TokenIdent>
> A solution could amend the Hive specific extension to support:
> * Pluggable delegation token and master key store (ZooKeeper as alternative for in-memory
AbstractDelegationTokenSecretManager)
> * Delegation token retrieval from token store when not found in memory (wrap/extend retrievePassword(...))
> * Cancellation of token in token store
> * Purging of expired tokens from token store
> http://www.mail-archive.com/hcatalog-user@incubator.apache.org/msg00053.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message