atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashutosh Mestry <ames...@hortonworks.com>
Subject Re: Review Request 58434: Import API: Improved the API to Update Existing Types Attributes
Date Fri, 16 Jun 2017 17:55:54 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58434/
-----------------------------------------------------------

(Updated June 16, 2017, 5:55 p.m.)


Review request for atlas and Madhan Neethiraj.


Changes
-------

Includes:
- Addressed review comments.
- Refactored _ImportService_: Moved _typeDefinition_ processing to separate class. 
- Added option to _AtlasImportRequest_ to choose typeDef import.
- Added unit tests to verify the new option.


Bugs: ATLAS-1734
    https://issues.apache.org/jira/browse/ATLAS-1734


Repository: atlas


Description
-------

**Background**
Existing version of Import API allows for importing types that are not already present in
the system being imported in. This causes import to fail in the cases where the data being
imported happens to have the additional attribute.

**Approach**
New class _TypeAttributeDifference_ does a attribute diff betweeing 2 types.
_ImportService.processTypeDefs_ method now includes additional step for attribute updation.
Metrics have been updated to reflect the updates.

**CURL**

The ZIP files mentioned below can be found at: _repository/src/test/resources/_


Create a file _importOptions.json_ with contents below:
```javascript
{ 
  "options": {
    "updateAttributes": true
    }
}
```

Use the following curl to perform an import with default attributes.

```javascript
curl -g -X POST -u admin:admin -H "Content-Type: multipart/form-data" -H "Cache-Control: no-cache"
-F data=@salesNewTypeAttrs.zip "http://localhost:21000/api/atlas/admin/import"
```
Note the attributes of the tables. 

Then perform this import with the new option.
```javascript
curl -g -X POST -u admin:admin -H "Content-Type: multipart/form-data" -H "Cache-Control: no-cache"
-F request=@importOptions.json -F data=@salesNewTypeAttrs-next.zip "http://localhost:21000/api/atlas/admin/import"
```
**Impact to API Consumer**
None.


Diffs (updated)
-----

  intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportRequest.java b19f7097 
  repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java 7744c781

  repository/src/main/java/org/apache/atlas/repository/impexp/ImportService.java 4ffbb88c

  repository/src/main/java/org/apache/atlas/repository/impexp/ImportTypeDefProcessor.java
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/repository/impexp/TypeAttributeDifference.java
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java
75e91320 
  repository/src/test/java/org/apache/atlas/repository/impexp/AtlasImportRequestTest.java
0844bf0d 
  repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java a6fed628

  repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTestUtils.java
PRE-CREATION 
  repository/src/test/java/org/apache/atlas/repository/impexp/TypeAttributeDifferenceTest.java
PRE-CREATION 
  repository/src/test/java/org/apache/atlas/repository/impexp/UniqueListTest.java PRE-CREATION

  repository/src/test/resources/salesNewTypeAttrs-next.zip PRE-CREATION 
  repository/src/test/resources/salesNewTypeAttrs.zip PRE-CREATION 


Diff: https://reviews.apache.org/r/58434/diff/3/

Changes: https://reviews.apache.org/r/58434/diff/2-3/


Testing
-------

**Unit tests**
- _ImportServiceTest_ Updated to handle the new case. New light-weight _zip_ files added test
resources to verify the functionality.
- _TypeAttributeDifferenceTest_ Unit test for _TypeAttributeDifference_ class.
- Additional unit tests: _UniqueList_.
- Import-Export APIs verified with test file set.


Thanks,

Ashutosh Mestry


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message