atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Madhan Neethiraj <mad...@apache.org>
Subject Re: Review Request 56795: Import REST API
Date Fri, 17 Feb 2017 23:15:42 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56795/#review165997
-----------------------------------------------------------


Ship it!




Ship It!

- Madhan Neethiraj


On Feb. 17, 2017, 7:57 p.m., Ashutosh Mestry wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56795/
> -----------------------------------------------------------
> 
> (Updated Feb. 17, 2017, 7:57 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-1503
>     https://issues.apache.org/jira/browse/ATLAS-1503
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> The patch includes the optimization done for import:
> - Handling of large databases (151 tables, each with 80 columns)
> - Improved handling of zip file inputs.
> - Improved metrics data. This now includes granular breakdown of individual entities
and list of all entities imported.
> 
> Import is implemented in 2 flavors:
> - ZIP file upload in POST.
>    CURL:
>    ```
>    curl -X POST -u admin:admin -H "Content-Type: application/octet-stream" -H "Cache-Control:
no-cache" --data-binary @../docs/DefaultDB-Stocks.zip "http://localhost:21000/api/atlas/admin/import"
>    ```
> - Specify ZIP file as part of query parameter in GET.
>    CURL:
>    ```
>    curl -X POST -u admin:admin -H "Cache-Control: no-cache" "http://localhost:21000/api/atlas/admin/importFile?FILENAME=/Users/amestry/Documents/DefaultDB-Sales.zip"
>    ```
> 
> 
> Diffs
> -----
> 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 584bf25 
>   intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportResult.java a5eeef1 
>   intg/src/main/java/org/apache/atlas/model/instance/EntityMutations.java 74e3c57 
>   repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java
1cd4375 
>   repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java
4312287 
>   repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStream.java
010b626 
>   repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStreamForImport.java
PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
2e71ab8 
>   webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java c8c0099 
>   webapp/src/main/java/org/apache/atlas/web/resources/ExportService.java 7d3d442 
>   webapp/src/main/java/org/apache/atlas/web/resources/ImportService.java 7554cdb 
>   webapp/src/main/java/org/apache/atlas/web/resources/ZipSource.java ea62862 
> 
> Diff: https://reviews.apache.org/r/56795/diff/
> 
> 
> Testing
> -------
> 
> Status:
> - Large DB (151 tables): Export.
> - - Quick start databases: Done.
>   - Includes lineage information.
> - hive_db database with upload from CSV file: Done.
> - Unit tests: Pending.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message