hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vihang Karajgaonkar <vih...@cloudera.com>
Subject Review Request 62453: HIVE-17371 : Move tokenstores to metastore module
Date Fri, 22 Sep 2017 15:50:36 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62453/
-----------------------------------------------------------

Review request for hive, Aihua Xu, Alan Gates, and Thejas Nair.


Bugs: HIVE-17371
    https://issues.apache.org/jira/browse/HIVE-17371


Repository: hive-git


Description
-------

HIVE-17371 : Move tokenstores to metastore module

I think it makes more sense to move the tokenstore implementations to metastore instead of
having two copies of each in Hive and metastore. The TokenManager and SecretManager classes
can be kept as is to each consuming applications like HS2 or Metastore itself so that future
modifications can be done independently.
Here is the approach I followed.
1. Moved the HiveDelegationTokenManager and TokenStoreDelegationTokenSecretManager from shims-common
to service module since Metastore anyways has its own copy of TokenManager and Metastore cannot
have a dependency on shims-common.
2. Moved the MemoryTokenStore, ZooKeeperTokenStore and DBTokenStore to standalone-metastore
module. In order for these tokenstores to be consumable for multiple applications like HS2
and Metastore (and hopefully other applications in the future) we need to operate of higher
level abstraction of AbstractDelegationTokenIdentifier and DelegationTokenInformation.
3. Each application can define its DelegationTokenIndentifier like Hive already does in DelegationTokenIdentifier
(may be we should rename this to HiveDelegationTokenIdentifier to make it more explicit. Similarly,
metastore has its own implementation of DelegationTokenIdentifier in DelegationTokenIdentifier
(again we should rename it possibly to say MetastoreDelegationTokenIndentifier)
4. Modified the shims-common classes of DelegationTokenSecretManager and DelegationTokenStore
to operate on hadoop level abstractions instead of Hive defined ones.


Diffs
-----

  itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/TestHiveAuthFactory.java e3a51909df14d90a538426163f332a6378f35f40

  itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/TestJdbcWithDBTokenStore.java
d690aaa673a50785561750f4f461ec867b6f0abc 
  itests/hive-unit-hadoop2/src/test/java/org/apache/hadoop/hive/metastore/security/TestHadoopAuthBridge23.java
ef360400a1873ab142cb9cbdce045cbed394af0e 
  itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestDBTokenStore.java 4bfa22419cb1ea5abe56773c1c76186b01996e03

  itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestZooKeeperTokenStore.java
7800416a60239246e96497229d592736f2a9d79d 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java cf33cca24ffc7583f2be62bb79815b5e43b2ff42

  service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java 541fe5ec8da9ba66ac44c65e6fe0dcd3c601943f

  shims/common/src/main/java/org/apache/hadoop/hive/shims/Utils.java 3c93186082ae363497dc94ca1303db4f746fd8b5

  shims/common/src/main/java/org/apache/hadoop/hive/thrift/DBTokenStore.java d6dc0796e77591d3afca8dbd29c3aa0eff255dd0

  shims/common/src/main/java/org/apache/hadoop/hive/thrift/DelegationTokenSecretManager.java
5299e18743aa45c539287b335f95e8ce8df0fc35 
  shims/common/src/main/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java 867b4ed98859f681770fccfe7478ab519c823924

  shims/common/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java 6c8b362270e48cb5641d77b09f1a19d875df0cbb

  shims/common/src/main/java/org/apache/hadoop/hive/thrift/HiveDelegationTokenManager.java
b3e4a7608282be603e79d1d101679e239a5219b0 
  shims/common/src/main/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java 9d837b8fa19ed53546c4a95944484cc1f06a21e6

  shims/common/src/main/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java
4719b85e8d2f9045ae61e0e86c6624b865aa2653 
  shims/common/src/main/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java 885ec56d543e1c46e74dc148c73f249803b7604c

  standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/security/DelegationTokenStore.java
0cafeff89614c94ba551a5f1ba9c4d892ee5720a 
  standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/security/MemoryTokenStore.java
c484cd3132d08c8c0493fa98e79fa449e84ef9b8 
  standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/security/MetastoreDelegationTokenManager.java
2b0110fe0badcb571144067539031fb6ac81276b 
  standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/security/TokenStoreDelegationTokenSecretManager.java
4abcec789476982389b3c0bc34d4990c59dbf9da 
  standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/utils/SecurityUtils.java
9f0ca829b1e3a469a3275c54a74d90278fc030fa 


Diff: https://reviews.apache.org/r/62453/diff/1/


Testing
-------

HiveQA


Thanks,

Vihang Karajgaonkar


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message