atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Hagelberg <jnhagelb...@us.ibm.com>
Subject Re: Review Request 56283: ATLAS-746: After updating a set of entities, response contains only the first entity definition
Date Tue, 07 Feb 2017 04:17:07 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56283/
-----------------------------------------------------------

(Updated Feb. 7, 2017, 4:17 a.m.)


Review request for atlas and David Kantor.


Changes
-------

Rebased changes, fixed issues with EntityV2JerseyResourceIT


Bugs: ATLAS-746
    https://issues.apache.org/jira/browse/ATLAS-746


Repository: atlas


Description
-------

When a an entity is being created that contains nested entities, it is not currently possible
to determine what guids were assigned to the nested entities.  This is because the result
going back to the caller is simply a flat list of the guids of the entities that were created.
 We get back the guids, but it is impossible to know which nested entities they refer to without
executing an additional query.

This task addresses this issue by adding a "guidAssignments" section to REST API output when
creating / updating entities.  This has a map from the "unassigned" guids in the original
request to the guids that were assigned during the entity creation.  This makes it easy to
determine what the guid is for all the entities that were created, regardlesss of how deeply
nested they are.

Example:

{ 
   "entitiesMutated":{ 
 	…
   },
   "guidAssignments":{ 
      "-267745036104320":"27423408-4619-4736-9b99-b9f2bb5783b7",
      "-267745036104310":"41628d3d-6da0-4d79-9f6b-863c6631d31f",
      "-267745036104311":"b75acf2d-0523-423e-ac1e-4721b2322b91",
      "-267745036104300":"dbbd27b1-048a-447c-82dd-7ccff717bd4a",
      "-267745036104309":"cc12c71f-6d30-499e-8dab-f243b8547f42",
      "-267745036104316":"f5149832-c3ef-47ae-a982-a29b365e0199",
      "-267745036104305":"fcdd4076-c3e6-4a4c-ac56-c62599ae79c9",
      "-267745036104306":"7cba46ff-52ca-49e6-8327-ab5f7892deba",
      "-267745036104317":"b33aea18-365a-4017-9659-a084593dd47e",
      "-267745036104318":"d1c36c36-0968-4758-b2f9-d41ed6097c1b",
      "-267745036104307":"1da32314-7ac2-4d88-97c9-a1f26edb281b",
      "-267745036104319":"50c707ce-53e2-49c6-862b-b79a37102ae5",
      "-267745036104308":"c49655bc-ba54-4e37-94f2-938c9bc565a5",
      "-267745036104301":"f6190422-09f7-4f17-bda3-a1b7a5f5415d",
      "-267745036104312":"b6b44a3f-6307-4d67-a777-a29764971dba",
      "-267745036104302":"e3f570b7-9b74-43f4-ba6e-c730d8a630b3",
      "-267745036104313":"5d6b033f-eec3-4bd6-b655-be48042a6530",
      "-267745036104314":"0ca94243-9187-43a1-aaad-ced0ea2a1731",
      "-267745036104303":"56b42729-45af-44e5-98d4-a6a77889340a",
      "-267745036104315":"8061b59b-fa1c-4ca8-8f18-72c3bcc82d73",
      "-267745036104304":"b9b3ecf7-47f6-4887-9cd7-32febf0e2fa2"
   }
}


I also fixed an issue preventing nested entities from being created.  The issue there was
that the mapping composite attribute representation in the old and new format was inconsistent.
 I've changed the code so that an attribute is treated as composite if it is either a "mappedFromRef"
attribute or it has a foreign key with "update"as the onDelete action.  There were two competing
ways of determining this.

One quick note about EntityV2JerseyResourceIT.testSubmitEntityWithBadDateFormat().  This test,
on the surface, looked like it was testing that creating an entity with a date in an invalid
format would fail.  In reality, though, the invalid date was being nulled out and ignored.
 The failure was actually happening after that point, because the database reference ("db")
could not be resolved.  It was failing with this exception: 

org.apache.atlas.typesystem.exception.EntityNotFoundException: Could not find vertex for Class
Reference (type: hive_db_v2, id: <unassigned>))

This was a direct result of "db" being a non-composite attribute : ClassType.convert() replaces
non-composite reference attributes in an entity being created with their Id.

As part of the testing for these changes, I changed the "db" attribute to be a composite attribute.
 As a result, the entity is now able to be created successsfully.  I've updated the test accordingly.


Diffs (updated)
-----

  catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java 726351aa2037c3ead59eaa3fe1d052cc0889f525

  client/src/main/java/org/apache/atlas/AtlasBaseClient.java 9b699914bd4870683f271c2bab0e3f87ef76fb9a

  client/src/main/java/org/apache/atlas/AtlasClient.java 154644dd9e637d612960a8c811c54efaf18d3b27

  client/src/main/java/org/apache/atlas/AtlasEntitiesClientV2.java 2b3669dde8a67e054675ea44a90c281ffecc32af

  client/src/main/java/org/apache/atlas/CreateUpdateEntitiesResult.java PRE-CREATION 
  intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java 874a43ab9730b23958022e4e64c346e0925ef5cd

  intg/src/main/java/org/apache/atlas/model/instance/GuidMapping.java PRE-CREATION 
  repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java 1d61ea84f33e8a273229cc077cf1c205d7e29f96

  repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java
27bf6d7278ac22ae9d1b572141e5a19e1293c48f 
  repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
1a88251f5c465fca6bd9a3dd28dc9afb3540fe25 
  repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java
d48c87e2630f81125192ae89bce2b32c2b8ff5ff 
  repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasTypeDefGraphStoreV1.java
88197aca7966cb1a0d8f5b33bedeb9ff27689b52 
  repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java b14531f0b994674daa72773e65476151215d664b

  repository/src/test/java/org/apache/atlas/BaseRepositoryTest.java 9714a8b2ba3edee65632352d9059dd94374faaae

  repository/src/test/java/org/apache/atlas/TestUtils.java 1d1a5e04210ccc48ec989c514d3796c2d7dc211c

  repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java
ce87c9ee5ef45dc8339145e40e9e2a6c37a40e9f 
  repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
d6136edc7cf29d648292ce6b852e88081c1c3165 
  repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java
96b017364718312b55715a1056a5e43155c9152a 
  repository/src/test/java/org/apache/atlas/repository/graph/GraphHelperTest.java f5a6a0519b9153e6edc68ed4502dad2c15a5a1e0

  repository/src/test/java/org/apache/atlas/repository/graph/GraphRepoMapperScaleTest.java
a03f965f81dadd932bb12c6c6ab62fc7ebb27322 
  repository/src/test/java/org/apache/atlas/repository/typestore/GraphBackedTypeStoreTest.java
265b316a8582172b045f4313b63d3f4cc7f1ad72 
  repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java 1ad79b1f3e78f712905d57f58e1d00d59d535b70

  server-api/src/main/java/org/apache/atlas/services/MetadataService.java d5d8d9b5d57f4bf7fbf49373337d526d67583e5e

  typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java 1dcad1408e08ff049444cbbe72668efe9b80e55e

  webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 1559eb91d89e610f96f0c2ddbf6ad1ce7095773f

  webapp/src/main/java/org/apache/atlas/util/RestUtils.java cecf636a0b3e3c9c9806497768c0986d773f1c5b

  webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java 7f5a05696f415eb56ffa90dedc3d6708a921b046

  webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 9ca684d956724512f787ff5169eece70197bd0b1

  webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java af0377e9f9c52a52b4f3ebc3f6fa13762f5569d3

  webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java 9c0ccf66671ec09524b34b29bb563959db0659ef

  webapp/src/test/java/org/apache/atlas/examples/QuickStartV2IT.java 952813917cd7a519bc3e6e2ae7f30c988ad9db91

  webapp/src/test/java/org/apache/atlas/util/RestUtilsTest.java PRE-CREATION 
  webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 18bbc3bc3e26c66885b652d9d4efdc538c31f0cc

  webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java 9724c4c4bb2dc6c209cdf643d0eb4eb4856eb981

  webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java 29a6d4913e3742ea54cbd4d23ca8ba439c48b23a


Diff: https://reviews.apache.org/r/56283/diff/


Testing
-------

Ran full build, no regressions found


Thanks,

Jeff Hagelberg


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message