atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apoorv Naik <naik.apo...@gmail.com>
Subject Re: Review Request 56184: ATLAS-1499: Notification processing using V2 Store
Date Fri, 17 Feb 2017 07:25:12 GMT


> On Feb. 17, 2017, 6:43 a.m., Madhan Neethiraj wrote:
> > webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java,
line 246
> > <https://reviews.apache.org/r/56184/diff/8/?file=1638276#file1638276line246>
> >
> >     userName set in atlasClient is used in audit-record that gets saved to HBase
audit table. How is this handled in v2 approach (of calling entity store)?
> >     
> >     Perhaps AtlasEntityStore APIs need to be updated to get the username from the
caller? Let's address this in a separate JIRA.

The user gets set in the RequestContext which is then used by the EntityAuditListener for
logging audit event (good catch)


> On Feb. 17, 2017, 6:43 a.m., Madhan Neethiraj wrote:
> > webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java,
line 272
> > <https://reviews.apache.org/r/56184/diff/8/?file=1638276#file1638276line272>
> >
> >     wouldn't this overwrite the value of the unique-attribute which would have been
updated in the entity?
> >     
> >     A better approach would be to get the guid and update entity's guid:
> >     
> >         AtlasEntityType entityType = typeRegistry.getEntityTypeByName(entity.getTypeName());
> >     
> >         Map<String, Object> uniqAttributes = new HashMap<String, Object>()
{{
> >                                         put(partialUpdateRequest.getAttribute(),
partialUpdateRequest.getAttributeValue());
> >         }};
> >     
> >         AtlasVertex vertex = AtlasGraphUtilsV1.findByUniqueAttributes(entityType,
uniqAttributes);
> >         String      guid   = vertex != null ? AtlasGraphUtilsV1.getIdFromVertex(vertex)
: null;
> >     
> >         if (StringUtil.isNotBlank(guid)) {
> >             entity.setGuid(guid);
> >         }

This is exactly the updateByUniqueAttribute method in the store, let's just use that for now.


- Apoorv


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56184/#review165905
-----------------------------------------------------------


On Feb. 17, 2017, 6:01 a.m., Apoorv Naik wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56184/
> -----------------------------------------------------------
> 
> (Updated Feb. 17, 2017, 6:01 a.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Sarath Subramanian, Suma Shivaprasad, and
Vimal Sharma.
> 
> 
> Bugs: ATLAS-1499
>     https://issues.apache.org/jira/browse/ATLAS-1499
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> ATLAS-1499: Notification processing using V2 Store
> 
> 
> Diffs
> -----
> 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java ae6be843 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java edaede0c 
>   intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java 5e8ce351

>   repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListener.java
1ef803c0 
>   server-api/src/main/java/org/apache/atlas/RequestContextV1.java bf731746 
>   webapp/src/main/java/org/apache/atlas/LocalAtlasClient.java 2b714896 
>   webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java f2416810

>   webapp/src/main/java/org/apache/atlas/web/adapters/AtlasAbstractFormatConverter.java
f1f3d18a 
>   webapp/src/main/java/org/apache/atlas/web/adapters/AtlasArrayFormatConverter.java aa14aff4

>   webapp/src/main/java/org/apache/atlas/web/adapters/AtlasClassificationFormatConverter.java
dc740f55 
>   webapp/src/main/java/org/apache/atlas/web/adapters/AtlasEntityFormatConverter.java
cb1390d1 
>   webapp/src/main/java/org/apache/atlas/web/adapters/AtlasEnumFormatConverter.java 6d8e3aee

>   webapp/src/main/java/org/apache/atlas/web/adapters/AtlasFormatConverter.java a7157a36

>   webapp/src/main/java/org/apache/atlas/web/adapters/AtlasFormatConverters.java 968d74fe

>   webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java b1dae56d

>   webapp/src/main/java/org/apache/atlas/web/adapters/AtlasMapFormatConverter.java 6967c4f5

>   webapp/src/main/java/org/apache/atlas/web/adapters/AtlasObjectIdConverter.java 11a020d8

>   webapp/src/main/java/org/apache/atlas/web/adapters/AtlasPrimitiveFormatConverter.java
2b70c5e1 
>   webapp/src/main/java/org/apache/atlas/web/adapters/AtlasStructFormatConverter.java
920b48b0 
>   webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 2c2c16d7 
>   webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java 9518f540 
>   webapp/src/test/java/org/apache/atlas/LocalAtlasClientTest.java c5616dfe 
>   webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerKafkaTest.java
873e5625 
>   webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerTest.java
f06f7912 
> 
> Diff: https://reviews.apache.org/r/56184/diff/
> 
> 
> Testing
> -------
> 
> mvn clean install -DskipITs executed successfully.
> 
> UI/Manual testing details
> 
> **Changes seen on UI **
> 1. Manually created table in hive, showed up in graph and as an entity from the search
UI
> 2. Created a view from the above table and the lineage showed up in atlas dashboard
> 3. added a column to the above table
> 4. altered table columns
> 
> 
> Will be running HiveHookIT to verify more cases.
> 
> **Update**
> ```java 
> Tests run: 43, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 393.359 sec <<<
FAILURE! - in org.apache.atlas.hive.hook.HiveHookIT
> testAlterTableRename(org.apache.atlas.hive.hook.HiveHookIT)  Time elapsed: 0.459 sec
 <<< FAILURE!
> org.apache.atlas.AtlasServiceException: Metadata service API org.apache.atlas.AtlasBaseClient$APIInfo@7856f41a
failed with status 400 (Bad Request) Response Body ({"error":"PII_Trait\u2E07\u150E\u1D79\u36B9\uF575\u722B\u5841\u71C5\uC0C5\uE1CA:
invalid name for CLASSIFICATION.  Names must consist of a letter followed by a sequence of
letter, number, or _ characters"})
>         at org.apache.atlas.hive.hook.HiveHookIT.createTrait(HiveHookIT.java:946)
>         at org.apache.atlas.hive.hook.HiveHookIT.testAlterTableRename(HiveHookIT.java:889)
> 
> testCreateTable(org.apache.atlas.hive.hook.HiveHookIT)  Time elapsed: 10.485 sec  <<<
FAILURE!
> java.lang.AssertionError: expected:<1487308686000> but was:<1487279886000>
>         at org.apache.atlas.hive.hook.HiveHookIT.verifyTimestamps(HiveHookIT.java:189)
>         at org.apache.atlas.hive.hook.HiveHookIT.testCreateTable(HiveHookIT.java:170)
> 
> testTraitsPreservedOnColumnRename(org.apache.atlas.hive.hook.HiveHookIT)  Time elapsed:
5.501 sec  <<< FAILURE!
> org.apache.atlas.AtlasServiceException: Metadata service API org.apache.atlas.AtlasBaseClient$APIInfo@5b3eb1dd
failed with status 400 (Bad Request) Response Body ({"error":"PII_Trait\u381C\uBA9E\u605E\u08CF\uD4E0\u151B\uEF7E\uA387\uBCE4\u4ECF:
invalid name for CLASSIFICATION.  Names must consist of a letter followed by a sequence of
letter, number, or _ characters"})
>         at org.apache.atlas.hive.hook.HiveHookIT.createTrait(HiveHookIT.java:946)
>         at org.apache.atlas.hive.hook.HiveHookIT.testTraitsPreservedOnColumnRename(HiveHookIT.java:1255)
> 
> 
> Results :
> 
> Failed tests: 
>   HiveHookIT.testAlterTableRename:889->createTrait:946 » AtlasService Metadata s...
>   HiveHookIT.testCreateTable:170->verifyTimestamps:189 expected:<1487308686000>
but was:<1487279886000>
>   HiveHookIT.testTraitsPreservedOnColumnRename:1255->createTrait:946 » AtlasService
> ```
> 
> Test failures due to recent naming restrictions, need to update IT code as well (out
of scope)
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message