atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sarath Subramanian <sar...@apache.org>
Subject Re: Review Request 67586: Export: Support for Incremental Export
Date Wed, 08 Aug 2018 18:38:56 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67586/#review206990
-----------------------------------------------------------


Fix it, then Ship it!




Ship It!


repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java
Lines 619 (patched)
<https://reviews.apache.org/r/67586/#comment290150>

    getLasModifiedTimestamp => getLastModifiedTimestamp


- Sarath Subramanian


On Aug. 8, 2018, 10:02 a.m., Ashutosh Mestry wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67586/
> -----------------------------------------------------------
> 
> (Updated Aug. 8, 2018, 10:02 a.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Madhan Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-2738
>     https://issues.apache.org/jira/browse/ATLAS-2738
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> **Approach**
> Existing _ExportService_ has been extended.
> 
> _AtlasExportRequest_ supports: _fetchType_ as _incremental_. This takes additional parameter
_fromTimestamp_ of type long. 
> 
> During export process, the entities that are encountered added to _ZipSink_ only when
their _modifiedTimestamp_ is less than the _fromTimestamp_ value. 
> 
> This option can thus be applied with all the existing _matchType_.
> 
> **Updates to AtlasClient**
> _AtlasClient_ now support _exportData_ API. This allows for _ExportService_ to be accessed
via client APIs.
> _AtlasClient_ has additional _importData_ API. This takes _byte_ array as parameter,
thus allowing imports without needing absolute file name.
> 
> **Documentation**
> None.
> 
> 
> Diffs
> -----
> 
>   intg/src/main/java/org/apache/atlas/model/impexp/AtlasExportRequest.java 035216bc45e9c684bf9f49f13e5523570603d997

>   intg/src/main/java/org/apache/atlas/model/impexp/AtlasExportResult.java 4da91a041593eaef8449c641263052f05802e914

>   repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java eeb873511fcbf7392e9ebf7239cbb436165237cf

>   repository/src/test/java/org/apache/atlas/repository/impexp/ExportImportTestBase.java
d61b2747777fbfc0d94564a86d6117ab2fb182c2 
>   repository/src/test/java/org/apache/atlas/repository/impexp/ExportIncrementalTest.java
PRE-CREATION 
>   repository/src/test/java/org/apache/atlas/repository/impexp/ExportSkipLineageTest.java
5ae86ef3b6f057ac08368d86c445c53fa0a64641 
>   repository/src/test/java/org/apache/atlas/repository/impexp/ZipFileResourceTestUtils.java
f465d678d7d603918ef8f74e93dea462b523b6e4 
>   repository/src/test/resources/json/stocksDB-Entities/export-incremental.json PRE-CREATION

> 
> 
> Diff: https://reviews.apache.org/r/67586/diff/6/
> 
> 
> Testing
> -------
> 
> **Unit tests**
> Tests that exercise the new option added.
> 
> **Integration tests**
> New test added that perfom import and export with export option set to incremental.
> 
> 
> File Attachments
> ----------------
> 
> Export Option set to Incremental
>   https://reviews.apache.org/media/uploaded/files/2018/06/13/e1822097-c386-4618-90c9-de5116dddebb__export-incremental.json
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message