atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashutosh Mestry <ames...@hortonworks.com>
Subject Re: Review Request 65790: Data Migration Utility: Export
Date Mon, 05 Mar 2018 05:39:39 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65790/
-----------------------------------------------------------

(Updated March 5, 2018, 5:39 a.m.)


Review request for atlas, Apoorv Naik, Madhan Neethiraj, and Ruchi Solani.


Changes
-------

Updates include: 
- Addressed review comments. 
- Attached latest migration utility.


Bugs: ATLAS-2461
    https://issues.apache.org/jira/browse/ATLAS-2461


Repository: atlas


Description
-------

**Background**
The data migration utility allows for exporting data from an Atlas instance without the server
running. This is important as it will prevent Atlas from processing any requests.

**Approach**
The migration is a new utility that will perform export of data using a pre-defined export
request file.

The approach used by this application:
- Create an application context (_migrationContext.xml_). This context, prevents instantiation
of number of classes, most notably _webapp_, _notifications_, _listeners_.
- Create _ImportService_ using Spring's framework classes.

Here are the pieces:
- _MigrationApp_: Contains _main_. It is the entry point for the app.
- _Exporter_: Contains plumbing needed to use the new _migrationContext.xml_ and create _ApplicationContext_
for use.
- Dengenerate classes _EmptyNotification_, _EmptyNotificationChangeListener_. These are necessary
to launch the application in a way that no notifications are sent out.
- _atlas_migration.py_ Creates environment for the app to execute. It launches the application.
- _migration-export-request.json_ This can be modified for environments that do not have latest
improvements to _Export_.
- _README_: Instructions on usage.

**Build**
The project maven's assembly building plugin. The build will create a ZIP file with the necessary
files.

_mvn install package_ generates a ZIP file in _target_ directory.

**Usage (from README)**
- Use Ambari to turn shutdown Atlas.
- Unzip contents of the ZIP to a directory on the server using: 'unzip atlas-migration-kit.zip'
- cd atlas-migration-kit
- Run 'python atlas_migration.py'. Use 'python atlas_migration.py <path-file.zip>' to
export to a specific file.
- To watch the progress: 'tail -f /var/log/atlas/application.log'.


Diffs (updated)
-----

  pom.xml 7db1be78 
  tools/atlas-migration-utility/pom.xml PRE-CREATION 
  tools/atlas-migration-utility/src/assembly/bin.xml PRE-CREATION 
  tools/atlas-migration-utility/src/main/java/org/apache/atlas/migration/Exporter.java PRE-CREATION

  tools/atlas-migration-utility/src/main/java/org/apache/atlas/migration/NoOpNotification.java
PRE-CREATION 
  tools/atlas-migration-utility/src/main/java/org/apache/atlas/migration/NoOpNotificationChangeListener.java
PRE-CREATION 
  tools/atlas-migration-utility/src/main/resources/README PRE-CREATION 
  tools/atlas-migration-utility/src/main/resources/atlas_migration.py PRE-CREATION 
  tools/atlas-migration-utility/src/main/resources/migration-export-request.json PRE-CREATION

  tools/atlas-migration-utility/src/main/resources/migrationContext.xml PRE-CREATION 


Diff: https://reviews.apache.org/r/65790/diff/5/

Changes: https://reviews.apache.org/r/65790/diff/4-5/


Testing
-------

**Unit tests**
None.

**Functional tests**
Exports from existing clusters.


File Attachments (updated)
----------------

Migration Utility
  https://reviews.apache.org/media/uploaded/files/2018/03/05/4d42eda3-cba8-441d-8505-12271cffc569__atlas-migration-kit-0.8.3-SNAPSHOT-bin.zip


Thanks,

Ashutosh Mestry


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message