atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashutosh Mestry <ames...@hortonworks.com>
Subject Re: Review Request 67641: ATLAS-2760: updated Hive hook to create S3 entities for references to S3 path
Date Tue, 19 Jun 2018 17:18:08 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67641/#review205006
-----------------------------------------------------------




addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java
Lines 532 (patched)
<https://reviews.apache.org/r/67641/#comment287856>

    Consider creating a method say 'isPathS3'



addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java
Lines 533 (patched)
<https://reviews.apache.org/r/67641/#comment287857>

    Consider moving this logic into new method say 'processS3Path'.


- Ashutosh Mestry


On June 19, 2018, 7:01 a.m., Madhan Neethiraj wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67641/
> -----------------------------------------------------------
> 
> (Updated June 19, 2018, 7:01 a.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, Don Bosco Durai, Nixon Rodrigues,
Ramesh Mani, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-2760
>     https://issues.apache.org/jira/browse/ATLAS-2760
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> Updated Hive hook to create entities S3 entities of type aws_s3_bucket, aws_s3_pseudo_dir,
aws_s3_object for references to S3 paths
> 
> 
> Diffs
> -----
> 
>   addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java
ce032877c 
>   addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateTable.java
daf5c860e 
> 
> 
> Diff: https://reviews.apache.org/r/67641/diff/1/
> 
> 
> Testing
> -------
> 
> Verified that Hive data export to S3 creates relevant S3 entities (bucket, pseudo-dir)
and shows lineage between Hive table and S3 pesudo-dir.
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message