atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Graham Wallis (JIRA)" <j...@apache.org>
Subject [jira] [Created] (ATLAS-2774) Options for hard and soft delete of instances
Date Tue, 03 Jul 2018 15:38:00 GMT
Graham Wallis created ATLAS-2774:
------------------------------------

             Summary: Options for hard and soft delete of instances
                 Key: ATLAS-2774
                 URL: https://issues.apache.org/jira/browse/ATLAS-2774
             Project: Atlas
          Issue Type: New Feature
            Reporter: Graham Wallis


*Overview*

For Entities and Relationships, the following delete behaviors are desired.

Atlas can be configured to offer hard delete (only). Alternatively it can be configured to
offer soft-delete. Configuration is achieved by setting the delete handler configuration property
(see atlas-application.properties) to either the SoftDeleteHandlerV1 or the HardDeleteHandlerV1.
The default (for when the property is not set) is to use the SoftDeleteHandlerV1.

The AtlasEntityStoreV2 deleteById() and deleteByIds() and methods and the AtlasRelationshipStoreV2
deleteById() method are sensitive to the above configuration. When the configuration is soft
these methods will perform a soft delete; when it is hard they will perform a hard delete.

In addition to the above methods, the AtlasEntityStoreV2 and AtlasRelationshipStoreV2 offer
a (new) purgeById() method that ALWAYS performs a hard delete. This is true regardless of
which delete handler has been configured. When the configuration is hard, the purgeById()
methods and deleteById() methods are essentially equivalent. When the configuration is soft,
the purgeById() methods continue to provide a hard delete capability.

The AtlasEntityStoreV2 deleteById(), deleteByIds() and purgeById() methods will delete the
specified entities and any relationships to which they are connected. The AtlasRelationshipStoreV2
deleteById() and purgeById() methods will delete the specified relationship. Deletion of a
relationship may cause an upate to an entity to which the relationhsip is connected, if it
changes the propagation of classifications, for example.

In a hard delete or purge operation, an affected entity and relationship will be removed from
the graph and will not be returned in response to any future queries. This is true regardless
of whether the affected entity or relationship has already been soft deleted or not. i.e.
it's status could initially be ACTIVE or DELETED. Following the operation it will not exist.

In a soft delete, the affected entities and relationships (that initially have status ACTIVE)
are updated to set the status to DELETED. These instances can still be returned from queries
and searches if the appropriate control is selected (to include deleted instances). Without
that control selected they will not be returned.


*Use Case for soft delete*

Soft delete provides a 'safe' means of deleting instances from the repository whilst preserving
an audit trail and (if supported by the repository) enabling a deleted instance to be restored
(to ACTIVE status).


*Use Cases for hard delete*

One use case for hard delete is to permanently remove soft-deleted items some period of time
after they were soft deleted.
Another use case for hard delete is to is to correct (clean up after) a mistake in which a
potentially large batch of incomplete/corrupted/wrong metadata is loaded and needs to be fully
removed.


*Use Scenario for an OMAS user*

An OMAS provides a delete function that does NOT offer the user a choice of hard/soft. The
OMAS's delete function will invoke the relevant soft delete method provided by the OMRS -
deleteEntity or deleteRelationship. These are both soft delete methods.
 
The Atlas OMRS Connector's deleteEntity/deleteRelationship methods will check whether Atlas
is configured for hard or soft delete. If Atlas is configured for hard delete then soft-delete
is not possible, and the method will throw a FunctionNotSupported exception which is caught
by the OMAS.

On catching this exception the OMAS author should automatically escalate the soft delete to
a hard delete by calling either of the mandatory OMRS purgeEntity() or purgeRelationship()
methods. In the Atlas OMRS Connector these methods will invoke AtlasEntityStoreV2 purgeById()
or AtlasRelationshipStoreV2 purgeById(). The effect of this is that if soft delete is not
possible, a hard delete is used instead.

An OMAS does NOT provide a purge capability to a non-privileged user, but a suitably expert
OMRS or Atlas repository administrator can explicitly issue a purge call at the OMRS or Atlas
interface, by calling purgeEntity() or purgeRelationship() (or the relevant Atlas store's
purgeById() method).

 

*Use Scenario for an Administrator*

As highlighted above, a suitably expert repository Administrator can invoke the purgeEntity(),
purgeRelationship() methods directly. They may need to do this to permanently remove instances
following an earlier soft-delete and the timeout of a period of grace, or following a failed
import or batch update.

In addition to the above direct use by a repository administrator of the 'purge' methods,
an Administrator could alternatively use the deleteEntity(), deleteRelationship() methods
to perform soft deletes. The invoked method will behave as described above for an OMAS - performing
a soft-delete if possible and throwing a FunctionNotSupported exception otherwise. In the
latter case (no support for soft-delete), the Administrator COULD then choose to issue a purgeEntity(),
purgeRelationship() call.


*Restore of an entity or relationship*

If an instance has been deleted using a soft-delete, the object still exists but has a status
of 'DELETED'. Whilst in the DELETED state, the object can be:
* optionally included/excluded from search results
* restored using the OMRS method for restoreEntity or restoreRelationship.
* purged from the repository using the purgeEntity() or purgeRelationship() method (or the
underlying Atlas store purgeById() method).

To restore an entity or relationship that has been soft-deleted, an OMAS user or repository
administrator can use the restoreEntity() or restoreRealtionship() method of the OMRS connector.

In the case of the Atlas OMRS Connector, the restoreEntity() and restoreRealtionship() methods
invoke (new) Atlas store methods to update the affected entities and relationships, leaving
them in ACTIVE state and with classifications that reflect the propagation options between
all the resulting active entities and relationships.

 

*Within the Atlas Repository*

The AtlasEntityStoreV2 and AtlasRelationshipStoreV2 provide additional methods for purgeById()
that perform a hard delete, regardless of the choice of configured delete handler.

The stores also provide additional methods to enable restore of an entity or relationship
that has been soft-deleted.


*Within the Atlas OMRS Connector*

The Atlas OMRS Connector will interrogate the Atlas repository configuration to determine
whether it is configuraed for soft or hard deletes. The Atlas OMRS Connector implementations
of deleteEntity() and deleteRelationship() methods will therefore have prior knowledge as
to whether it is feasible to attempt to perform a soft-delete. If feasible, the soft-delete
will be issued to the relevant Atlas store. If not feasible, the methods will throw a FunctionNotSupported
exception.

Also within the Atlas OMRS Connector, the mandatory OMRS purgeEntity(), purgeRelationship()
methods will invoke the Atlas stores' purgeById() methods to perform permanent (hard) removal
of an entity or relationship from the repository. As outline above, these methods may be called
either by an OMAS that has attempted a soft-delete and been 'bounced' or called directly by
an Adminstrator.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message