atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Mestry (JIRA)" <j...@apache.org>
Subject [jira] [Created] (ATLAS-1851) Import API: Ability to Specify Starting Entity for Import
Date Wed, 31 May 2017 23:02:04 GMT
Ashutosh Mestry created ATLAS-1851:
--------------------------------------

             Summary: Import API: Ability to Specify Starting Entity for Import
                 Key: ATLAS-1851
                 URL: https://issues.apache.org/jira/browse/ATLAS-1851
             Project: Atlas
          Issue Type: Improvement
          Components:  atlas-core
    Affects Versions: trunk
            Reporter: Ashutosh Mestry
            Assignee: Ashutosh Mestry
             Fix For: trunk


h3. Background & Scenarios
Import operations have a potential to go on for several hours. It is likely that the the operation
does not complete and the server crashes. 

To improve on this situation, it would be beneficial to be able to specify the last successfully
imported entity using either the entity guid or the position in the entity creation order.
That way, the import operation could resume from a later point rather than starting from scratch.

h3. Implementation

_AtlasImportRequest_ should support 2 additional options:
* _startGuid_: Entity GUID that should be used to resume import operation.
* _startPosition_: Starting position of the entity within the _creationOrder_.

Within _ZipSource_ this option would be used to set the internal iterator to a position from
which the import can start. _ImportService_ could read do the needful before starting to process
entities.

h3. Additional Implementation Guidance
Logs generated during import operations do not have the necessary information necessary to
help the consumer with information needed to use these new options.

It would be worth while to include the entity index within _creationOrder_ and the guid of
the entity that was successfully imported.




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message