atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Madhan Neethiraj <mad...@apache.org>
Subject Re: Review Request 56184: ATLAS-1499: Notification processing using V2 Store
Date Fri, 17 Feb 2017 06:43:25 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56184/#review165905
-----------------------------------------------------------


Fix it, then Ship it!





webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java 
<https://reviews.apache.org/r/56184/#comment237738>

    userName set in atlasClient is used in audit-record that gets saved to HBase audit table.
How is this handled in v2 approach (of calling entity store)?
    
    Perhaps AtlasEntityStore APIs need to be updated to get the username from the caller?
Let's address this in a separate JIRA.



webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java (line 269)
<https://reviews.apache.org/r/56184/#comment237739>

    wouldn't this overwrite the value of the unique-attribute which would have been updated
in the entity?
    
    A better approach would be to get the guid and update entity's guid:
    
        AtlasEntityType entityType = typeRegistry.getEntityTypeByName(entity.getTypeName());
    
        Map<String, Object> uniqAttributes = new HashMap<String, Object>() {{
                                        put(partialUpdateRequest.getAttribute(), partialUpdateRequest.getAttributeValue());
        }};
    
        AtlasVertex vertex = AtlasGraphUtilsV1.findByUniqueAttributes(entityType, uniqAttributes);
        String      guid   = vertex != null ? AtlasGraphUtilsV1.getIdFromVertex(vertex) :
null;
    
        if (StringUtil.isNotBlank(guid)) {
            entity.setGuid(guid);
        }


- Madhan Neethiraj


On Feb. 17, 2017, 6:01 a.m., Apoorv Naik wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56184/
> -----------------------------------------------------------
> 
> (Updated Feb. 17, 2017, 6:01 a.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Sarath Subramanian, Suma Shivaprasad, and
Vimal Sharma.
> 
> 
> Bugs: ATLAS-1499
>     https://issues.apache.org/jira/browse/ATLAS-1499
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> ATLAS-1499: Notification processing using V2 Store
> 
> 
> Diffs
> -----
> 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java ae6be843 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java edaede0c 
>   intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java 5e8ce351

>   repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListener.java
1ef803c0 
>   server-api/src/main/java/org/apache/atlas/RequestContextV1.java bf731746 
>   webapp/src/main/java/org/apache/atlas/LocalAtlasClient.java 2b714896 
>   webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java f2416810

>   webapp/src/main/java/org/apache/atlas/web/adapters/AtlasAbstractFormatConverter.java
f1f3d18a 
>   webapp/src/main/java/org/apache/atlas/web/adapters/AtlasArrayFormatConverter.java aa14aff4

>   webapp/src/main/java/org/apache/atlas/web/adapters/AtlasClassificationFormatConverter.java
dc740f55 
>   webapp/src/main/java/org/apache/atlas/web/adapters/AtlasEntityFormatConverter.java
cb1390d1 
>   webapp/src/main/java/org/apache/atlas/web/adapters/AtlasEnumFormatConverter.java 6d8e3aee

>   webapp/src/main/java/org/apache/atlas/web/adapters/AtlasFormatConverter.java a7157a36

>   webapp/src/main/java/org/apache/atlas/web/adapters/AtlasFormatConverters.java 968d74fe

>   webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java b1dae56d

>   webapp/src/main/java/org/apache/atlas/web/adapters/AtlasMapFormatConverter.java 6967c4f5

>   webapp/src/main/java/org/apache/atlas/web/adapters/AtlasObjectIdConverter.java 11a020d8

>   webapp/src/main/java/org/apache/atlas/web/adapters/AtlasPrimitiveFormatConverter.java
2b70c5e1 
>   webapp/src/main/java/org/apache/atlas/web/adapters/AtlasStructFormatConverter.java
920b48b0 
>   webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 2c2c16d7 
>   webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java 9518f540 
>   webapp/src/test/java/org/apache/atlas/LocalAtlasClientTest.java c5616dfe 
>   webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerKafkaTest.java
873e5625 
>   webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerTest.java
f06f7912 
> 
> Diff: https://reviews.apache.org/r/56184/diff/
> 
> 
> Testing
> -------
> 
> mvn clean install -DskipITs executed successfully.
> 
> UI/Manual testing details
> 
> **Changes seen on UI **
> 1. Manually created table in hive, showed up in graph and as an entity from the search
UI
> 2. Created a view from the above table and the lineage showed up in atlas dashboard
> 3. added a column to the above table
> 4. altered table columns
> 
> 
> Will be running HiveHookIT to verify more cases.
> 
> **Update**
> ```java 
> Tests run: 43, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 393.359 sec <<<
FAILURE! - in org.apache.atlas.hive.hook.HiveHookIT
> testAlterTableRename(org.apache.atlas.hive.hook.HiveHookIT)  Time elapsed: 0.459 sec
 <<< FAILURE!
> org.apache.atlas.AtlasServiceException: Metadata service API org.apache.atlas.AtlasBaseClient$APIInfo@7856f41a
failed with status 400 (Bad Request) Response Body ({"error":"PII_Trait\u2E07\u150E\u1D79\u36B9\uF575\u722B\u5841\u71C5\uC0C5\uE1CA:
invalid name for CLASSIFICATION.  Names must consist of a letter followed by a sequence of
letter, number, or _ characters"})
>         at org.apache.atlas.hive.hook.HiveHookIT.createTrait(HiveHookIT.java:946)
>         at org.apache.atlas.hive.hook.HiveHookIT.testAlterTableRename(HiveHookIT.java:889)
> 
> testCreateTable(org.apache.atlas.hive.hook.HiveHookIT)  Time elapsed: 10.485 sec  <<<
FAILURE!
> java.lang.AssertionError: expected:<1487308686000> but was:<1487279886000>
>         at org.apache.atlas.hive.hook.HiveHookIT.verifyTimestamps(HiveHookIT.java:189)
>         at org.apache.atlas.hive.hook.HiveHookIT.testCreateTable(HiveHookIT.java:170)
> 
> testTraitsPreservedOnColumnRename(org.apache.atlas.hive.hook.HiveHookIT)  Time elapsed:
5.501 sec  <<< FAILURE!
> org.apache.atlas.AtlasServiceException: Metadata service API org.apache.atlas.AtlasBaseClient$APIInfo@5b3eb1dd
failed with status 400 (Bad Request) Response Body ({"error":"PII_Trait\u381C\uBA9E\u605E\u08CF\uD4E0\u151B\uEF7E\uA387\uBCE4\u4ECF:
invalid name for CLASSIFICATION.  Names must consist of a letter followed by a sequence of
letter, number, or _ characters"})
>         at org.apache.atlas.hive.hook.HiveHookIT.createTrait(HiveHookIT.java:946)
>         at org.apache.atlas.hive.hook.HiveHookIT.testTraitsPreservedOnColumnRename(HiveHookIT.java:1255)
> 
> 
> Results :
> 
> Failed tests: 
>   HiveHookIT.testAlterTableRename:889->createTrait:946 » AtlasService Metadata s...
>   HiveHookIT.testCreateTable:170->verifyTimestamps:189 expected:<1487308686000>
but was:<1487279886000>
>   HiveHookIT.testTraitsPreservedOnColumnRename:1255->createTrait:946 » AtlasService
> ```
> 
> Test failures due to recent naming restrictions, need to update IT code as well (out
of scope)
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message