atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashutosh Mestry <ames...@hortonworks.com>
Subject Re: Review Request 65220: Import: Support for Types with Relationship Attributes
Date Fri, 19 Jan 2018 17:29:03 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65220/
-----------------------------------------------------------

(Updated Jan. 19, 2018, 5:29 p.m.)


Review request for atlas, Madhan Neethiraj and Sarath Subramanian.


Changes
-------

Updates include: Addressed review comments.


Bugs: ATLAS-2372
    https://issues.apache.org/jira/browse/ATLAS-2372


Repository: atlas


Description
-------

**Background**
After introduction of _relationshipAttributes_ the import of types with relationship attributes
started failing.

**Analysis**
Root cause: The _AtlasEntityGraphDiscoveryV1_ started using _relationshipAttribute_ to discover
entities. This caused more entities to be discovered than what was happening before. This
caused the single stream entity not to lookup entities correctly. Failures started occuring
when these additional entities started getting connected by edges.

Example: _hive_db_ has relationship attribute that now has relationship pointing to _hive_table_.
This caused many more entities to be discovered. This also has a side effect of bringing-in
way more entities than needed.

With relationshipAttributes, edges started receiving GUIDs. Import process did not handle
creating of edges with incoming guids. 

**Approach**
- The new _AtlasEntityGraphDiscoveryV1_ entitiy discovery approach using relationships is
not being used during import. Additional discussion are needed to guage validity of this approach.
- _EntityMutationContext_ now has additional getter to know of import is in progress.
- Edge creation (new method _AtlasRelationshipStoreV1.createEdgeWithGuid_) has been modified
to support edge creation using specific guid.


Diffs (updated)
-----

  repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java c5e6534c

  repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java
450fa875 
  repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
7474a8a4 
  repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityMutationContext.java
8a6a0e3f 
  repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java ab25faaa



Diff: https://reviews.apache.org/r/65220/diff/3/

Changes: https://reviews.apache.org/r/65220/diff/2-3/


Testing
-------

**Unit tests**
- Additional tests for testing entities with relationships.
- (To execute tests locally, place the attached zip at: _./repository/src/test/resources_

**Functional tests**
- Suite of tests that verify export-import-export and diff comparision for accuracy testing.


File Attachments
----------------

Additional resource
  https://reviews.apache.org/media/uploaded/files/2018/01/18/6edce43a-b1df-415a-98ea-1ab43359a9d7__stocks-rel-2.zip


Thanks,

Ashutosh Mestry


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message