atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Mestry (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (ATLAS-2906) Import Process: Transforms: Handling Custom Attributes Does Not Update
Date Thu, 04 Oct 2018 01:10:00 GMT

     [ https://issues.apache.org/jira/browse/ATLAS-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ashutosh Mestry updated ATLAS-2906:
-----------------------------------
    Attachment: ATLAS-2906-Allow-transforms-to-be-applied-when-entit.patch

> Import Process: Transforms: Handling Custom Attributes Does Not Update
> ----------------------------------------------------------------------
>
>                 Key: ATLAS-2906
>                 URL: https://issues.apache.org/jira/browse/ATLAS-2906
>             Project: Atlas
>          Issue Type: Bug
>            Reporter: Ashutosh Mestry
>            Assignee: Ashutosh Mestry
>            Priority: Major
>         Attachments: ATLAS-2906-Allow-transforms-to-be-applied-when-entit.patch
>
>
> *Background*
> The new transform framework looks at the entity holistically and computes attributes
as it sees fit.
> Updating the _hive_storagedesc.location_ if entity-level transform is present, does
not apply the transform.
> *Steps* *to Duplicate*
>  * Attempt to import a zip file with the following transform.
> {code:java}
> { "options": { "transformers": "[{\"conditions\":{\"__entity\":\"topLevel: \"},\"action\":{\"__entity\":\"ADD_CLASSIFICATION:
mycluster0_replicated\"}},{\"action\":{\"__entity.replicatedTo\":\"CLEAR:\",\"__entity.replicatedFrom\":\"CLEAR:\"}},{\"conditions\":{\"hive_db.clusterName\":\"EQUALS:
mycluster0\"},\"action\":{\"hive_db.clusterName\":\"SET: mycluster1\"}},{\"conditions\":{\"hive_storagedesc.location\":\"STARTS_WITH_IGNORE_CASE:
hdfs://localhost.localdomain\"},\"action\":{\"hive_storagedesc.location\":\"REPLACE_PREFIX:
= :hdfs://localhost.localdomain=hdfs://mycluster1\"}}]", "replicatedFrom": "SFO$clx" } }
> {code}
> Locate any entity of type _hive_storagedesc_ and note the _location._ It does not apply
the updated location.
> *Root Cause*
>  * The new transform framework re-computes the location based on cached data.
>  * Solution: Prevent caching of this attribute so that it honors the externally set transformation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message