ranger-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Madhan Neethiraj <mad...@apache.org>
Subject Re: Review Request 64189: TagSync should replace use of V1 Atlas APIs with V2 APIs for efficient tag-download from Atlas
Date Wed, 06 Dec 2017 21:32:32 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64189/#review193044
-----------------------------------------------------------




tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasNotificationMapper.java
Line 132 (original), 132 (patched)
<https://reviews.apache.org/r/64189/#comment271536>

    Consider skipping ENTITY_CREATE only when no tags are associated with the entity. Import
feature in Atlas can create entitied with tags, which will result in ENTITY_CREATE notificaiton.



tagsync/src/main/java/org/apache/ranger/tagsync/source/atlasrest/AtlasRESTTagSource.java
Lines 347 (patched)
<https://reviews.apache.org/r/64189/#comment271538>

    breaking up to super-type classifications should be handled outside of line #328 (empty
attribute check). With the current implementation, a derived tag with no attribute will result
in super-type tags getting not associated with the entity.



tagsync/src/main/java/org/apache/ranger/tagsync/source/atlasrest/RangerAtlasEntity.java
Lines 25 (patched)
<https://reviews.apache.org/r/64189/#comment271543>

    Consider marking the members as final.



tagsync/src/main/java/org/apache/ranger/tagsync/source/atlasrest/RangerAtlasEntityWithTags.java
Lines 34 (patched)
<https://reviews.apache.org/r/64189/#comment271541>

    Consider marking these members as final.



tagsync/src/main/java/org/apache/ranger/tagsync/source/atlasrest/RangerAtlasEntityWithTags.java
Lines 54 (patched)
<https://reviews.apache.org/r/64189/#comment271540>

    trait.getValuesMap() shouldn't be throwing exceptions. Please review.



tagsync/src/main/java/org/apache/ranger/tagsync/source/atlasrest/RangerAtlasEntityWithTags.java
Lines 61 (patched)
<https://reviews.apache.org/r/64189/#comment271539>

    Adding attribute-type with every tag-instance seems unnecessary. Consider getting the
tag-attribute-type from AtlasTypeRegistry, which would be available in case of tag-download
via REST.



tagsync/src/main/java/org/apache/ranger/tagsync/source/atlasrest/RangerAtlasHdfsEntity.java
Lines 25 (patched)
<https://reviews.apache.org/r/64189/#comment271544>

    Consider marking the members as final.


- Madhan Neethiraj


On Dec. 4, 2017, 6:28 p.m., Abhay Kulkarni wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64189/
> -----------------------------------------------------------
> 
> (Updated Dec. 4, 2017, 6:28 p.m.)
> 
> 
> Review request for ranger and Madhan Neethiraj.
> 
> 
> Bugs: RANGER-1897
>     https://issues.apache.org/jira/browse/RANGER-1897
> 
> 
> Repository: ranger
> 
> 
> Description
> -------
> 
> Currently tag-synchronization via REST API method uses Atlas V1 APIs, which requires
large number of calls from Ranger tag-sync to Atlas server. In environments having large number
of entities, this approach can take a long time to download tags from Atlas. Use of Atlas
V2 APIs would significantly improve the performance.
> 
> 
> Diffs
> -----
> 
>   plugin-atlas/src/main/java/org/apache/ranger/authorization/atlas/authorizer/RangerAtlasResource.java
f056f3e 
>   pom.xml 589cd6a 
>   src/main/assembly/tagsync.xml 0b17151 
>   tagsync/pom.xml 74ff155 
>   tagsync/src/main/java/org/apache/ranger/tagsync/process/TagSynchronizer.java b07cd34

>   tagsync/src/main/java/org/apache/ranger/tagsync/sink/tagadmin/TagAdminRESTSink.java
c34b6ea 
>   tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasEntityWithTraits.java
77dee01 
>   tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasHbaseResourceMapper.java
8b36a31 
>   tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasHdfsResourceMapper.java
06bff90 
>   tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasHiveResourceMapper.java
a359622 
>   tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasKafkaResourceMapper.java
09ae5d1 
>   tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasNotificationMapper.java
f007ae5 
>   tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasResourceMapper.java
8ececdf 
>   tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasResourceMapperUtil.java
40a639b 
>   tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasStormResourceMapper.java
4ed01ca 
>   tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasTagSource.java c382db0

>   tagsync/src/main/java/org/apache/ranger/tagsync/source/atlasrest/AtlasRESTTagSource.java
4e0ae90 
>   tagsync/src/main/java/org/apache/ranger/tagsync/source/atlasrest/AtlasRESTUtil.java
00a101e 
>   tagsync/src/main/java/org/apache/ranger/tagsync/source/atlasrest/RangerAtlasEntity.java
PRE-CREATION 
>   tagsync/src/main/java/org/apache/ranger/tagsync/source/atlasrest/RangerAtlasEntityWithTags.java
PRE-CREATION 
>   tagsync/src/main/java/org/apache/ranger/tagsync/source/atlasrest/RangerAtlasHdfsEntity.java
PRE-CREATION 
>   tagsync/src/test/java/org/apache/ranger/tagsync/process/TestHbaseResourceMapper.java
e990c28 
>   tagsync/src/test/java/org/apache/ranger/tagsync/process/TestHdfsResourceMapper.java
392b096 
>   tagsync/src/test/java/org/apache/ranger/tagsync/process/TestHiveResourceMapper.java
7fde91a 
>   tagsync/src/test/java/org/apache/ranger/tagsync/process/TestKafkaResourceMapper.java
3beb82f 
> 
> 
> Diff: https://reviews.apache.org/r/64189/diff/3/
> 
> 
> Testing
> -------
> 
> Tested in local VM.
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message