ranger-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abhay Kulkarni <akulka...@hortonworks.com>
Subject Review Request 64189: TagSync should replace use of V1 Atlas APIs with V2 APIs for efficient tag-download from Atlas
Date Thu, 30 Nov 2017 00:52:47 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64189/
-----------------------------------------------------------

Review request for ranger and Madhan Neethiraj.


Bugs: RANGER-1897
    https://issues.apache.org/jira/browse/RANGER-1897


Repository: ranger


Description
-------

Currently tag-synchronization via REST API method uses Atlas V1 APIs, which requires large
number of calls from Ranger tag-sync to Atlas server. In environments having large number
of entities, this approach can take a long time to download tags from Atlas. Use of Atlas
V2 APIs would significantly improve the performance.


Diffs
-----

  pom.xml 589cd6a 
  src/main/assembly/tagsync.xml 0b17151 
  tagsync/pom.xml 74ff155 
  tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasEntityWithTraits.java
77dee01 
  tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasHbaseResourceMapper.java
8b36a31 
  tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasHdfsResourceMapper.java
06bff90 
  tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasHiveResourceMapper.java
a359622 
  tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasKafkaResourceMapper.java
09ae5d1 
  tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasNotificationMapper.java
f007ae5 
  tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasResourceMapper.java 8ececdf

  tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasResourceMapperUtil.java
40a639b 
  tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasStormResourceMapper.java
4ed01ca 
  tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasTagSource.java c382db0

  tagsync/src/main/java/org/apache/ranger/tagsync/source/atlasrest/AtlasRESTTagSource.java
4e0ae90 
  tagsync/src/main/java/org/apache/ranger/tagsync/source/atlasrest/AtlasRESTUtil.java 00a101e

  tagsync/src/main/java/org/apache/ranger/tagsync/source/atlasrest/RangerAtlasEntity.java
PRE-CREATION 
  tagsync/src/main/java/org/apache/ranger/tagsync/source/atlasrest/RangerAtlasEntityWithTags.java
PRE-CREATION 
  tagsync/src/main/java/org/apache/ranger/tagsync/source/atlasrest/RangerAtlasHdfsEntity.java
PRE-CREATION 
  tagsync/src/test/java/org/apache/ranger/tagsync/process/TestHbaseResourceMapper.java e990c28

  tagsync/src/test/java/org/apache/ranger/tagsync/process/TestHdfsResourceMapper.java 392b096

  tagsync/src/test/java/org/apache/ranger/tagsync/process/TestHiveResourceMapper.java 7fde91a

  tagsync/src/test/java/org/apache/ranger/tagsync/process/TestKafkaResourceMapper.java 3beb82f



Diff: https://reviews.apache.org/r/64189/diff/1/


Testing
-------

Tested in local VM.


Thanks,

Abhay Kulkarni


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message