atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Péter Gergő Barna (JIRA) <j...@apache.org>
Subject [jira] [Commented] (ATLAS-1879) Updating classification removes some properties
Date Wed, 28 Jun 2017 08:34:00 GMT

    [ https://issues.apache.org/jira/browse/ATLAS-1879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16066140#comment-16066140
] 

Péter Gergő Barna commented on ATLAS-1879:
------------------------------------------

[~madhan.neethiraj], 
I understand the problem may be in the updateTypesDef method:
(repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java)


{noformat}
    @Override
    @GraphTransaction
    public AtlasTypesDef updateTypesDef(AtlasTypesDef typesDef) throws AtlasBaseException
{
        if (LOG.isDebugEnabled()) {
            LOG.debug("==> AtlasTypeDefGraphStore.updateTypesDef(enums={}, structs={},
classfications={}, entities={})",
                    CollectionUtils.size(typesDef.getEnumDefs()),
                    CollectionUtils.size(typesDef.getStructDefs()),
                    CollectionUtils.size(typesDef.getClassificationDefs()),
                    CollectionUtils.size(typesDef.getEntityDefs()));
        }

        AtlasTransientTypeRegistry ttr = lockTypeRegistryAndReleasePostCommit();

        // Translate any NOT FOUND errors to BAD REQUEST
        try {
            ttr.updateTypes(typesDef);
        } catch (AtlasBaseException e) {
            if (AtlasErrorCode.TYPE_NAME_NOT_FOUND == e.getAtlasErrorCode()) {
                throw new AtlasBaseException(AtlasErrorCode.BAD_REQUEST, e.getMessage());
            } else {
                throw e;
            }
        }

        AtlasTypesDef ret = updateGraphStore(typesDef, ttr);

        if (LOG.isDebugEnabled()) {
            LOG.debug("<== AtlasTypeDefGraphStore.updateTypesDef(enums={}, structs={},
classfications={}, entities={})",
                    CollectionUtils.size(typesDef.getEnumDefs()),
                    CollectionUtils.size(typesDef.getStructDefs()),
                    CollectionUtils.size(typesDef.getClassificationDefs()),
                    CollectionUtils.size(typesDef.getEntityDefs()));
        }

        return ret;

{noformat}



This method updates the TypeRegistry first, then the GraphStorage.
The TypeRegistry is updated with the user input, which does not necessarily contain all fields.

I figured, maybe the GraphStorage should be updated first, and then the TypeRegistry:

{noformat}
AtlasTypesDef ret = updateGraphStore(typesDef, ttr);
ttr.updateTypes(ret);

{noformat}

Unfortunately, it will cause a NullPointerException:

{noformat}
2017-06-26 19:42:59,802 ERROR - [pool-1-thread-7 - /api/atlas/v2/types/typedefs - 23dc280b-d4e9-4c98-9e80-5496b77a98e3:]
~ graph rollback due to exception  (GraphTransactionInterceptor:71)
java.lang.NullPointerException
        at org.apache.atlas.repository.store.graph.v1.AtlasStructDefStoreV1.toJsonFromAttribute(AtlasStructDefStoreV1.java:508)
        at org.apache.atlas.repository.store.graph.v1.AtlasStructDefStoreV1.updateVertexPreUpdate(AtlasStructDefStoreV1.java:448)
        at org.apache.atlas.repository.store.graph.v1.AtlasClassificationDefStoreV1.updateVertexPreUpdate(AtlasClassificationDefStoreV1.java:365)
        at org.apache.atlas.repository.store.graph.v1.AtlasClassificationDefStoreV1.updateByName(AtlasClassificationDefStoreV1.java:215)
        at org.apache.atlas.repository.store.graph.v1.AtlasClassificationDefStoreV1.update(AtlasClassificationDefStoreV1.java:184)
        at org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore.updateGraphStore(AtlasTypeDefGraphStore.java:793)
        at org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore.updateTypesDef(AtlasTypeDefGraphStore.java:372)
        at org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore$$FastClassBySpringCGLIB$$5226c80b.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
        at org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:60)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
        at org.apache.atlas.repository.store.graph.v1.AtlasTypeDefGraphStoreV1$$EnhancerBySpringCGLIB$$f4b1ae3b.updateTypesDef(<generated>)
        at org.apache.atlas.web.rest.TypesREST.updateAtlasTypeDefs(TypesREST.java:326)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
        at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
        at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
        at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
        at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
        at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
        at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
        at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
        at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
        at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
        at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
        at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
        at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
        at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
        at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
...

{noformat}


I made some tests and I found, that if TypeRegistry update is called both before and after
the GraphStorage update, than it will fix the issue:


{noformat}
        AtlasTransientTypeRegistry ttr = lockTypeRegistryAndReleasePostCommit();

        // Translate any NOT FOUND errors to BAD REQUEST
        try {
            ttr.updateTypes(typesDef);
        } catch (AtlasBaseException e) {
            if (AtlasErrorCode.TYPE_NAME_NOT_FOUND == e.getAtlasErrorCode()) {
                throw new AtlasBaseException(AtlasErrorCode.BAD_REQUEST, e.getMessage());
            } else {
                throw e;
            }
        }

        AtlasTypesDef ret = updateGraphStore(typesDef, ttr);
        ttr.updateTypes(ret);

{noformat}


I am still looking into this to find a more elegant solution, any comments or hints are welcome,
thx





> Updating classification removes some properties
> -----------------------------------------------
>
>                 Key: ATLAS-1879
>                 URL: https://issues.apache.org/jira/browse/ATLAS-1879
>             Project: Atlas
>          Issue Type: Bug
>          Components:  atlas-core
>    Affects Versions: 0.8-incubating
>            Reporter: Laura Ngo
>         Attachments: Atlas-1789.postman_collection.json
>
>
> * Created classification via POST. 
> * Updated via PUT
> * Lost properties
> POST http://127.0.0.1:21000/api/atlas/v2/types/typedefs
> {code}
> {
>  "classificationDefs": [{
>   "name": "test_classification_11",
>   "description": "",
>   "createdBy" : "admin",
>   "superTypes": [],
>   "attributeDefs": [{
>     "name" : "test_class_11",
>     "typeName" : "string",
>     "isOptional" : true,
>     "isUnique" : true,
>     "isIndexable" : true,
>     "cardinality": "SINGLE",
>     "valuesMinCount": 0,
>     "valuesMaxCount": 1
> }]
>  }],
>  "entityDefs": [],
>  "enumDefs": [],
>  "structDefs": []
> }
> {code}
> GET http://127.0.0.1:21000/api/atlas/v2/types/classification/name/test_classification_11
> {code}
> {
> "category": "CLASSIFICATION",
> "guid": "83162fe1-4bb4-4a87-b2b8-364e751a1265",
> "createdBy": "admin",
> "createTime": 1497485890857,
> "updateTime": 1497485890857,
> "version": 1,
> "name": "test_classification_11",
> "description": "",
> "typeVersion": "1.0",
> "attributeDefs": [
>   {
> "name": "test_class_11",
> "typeName": "string",
> "isOptional": true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1,
> "isUnique": true,
> "isIndexable": true
> }
> ],
> "superTypes": [],
> }
> {code}
> PUT http://127.0.0.1:21000/api/atlas/v2/types/typedefs
> Update attribute.
> GET http://127.0.0.1:21000/api/atlas/v2/types/classification/name/test_classification_11
> {code}
> {
> "category": "CLASSIFICATION",
> "createdBy": "admin",
> "name": "test_classification_11",
> "description": "",
> "attributeDefs": [
>   {
> "name": "test_class_11",
> "typeName": "string",
> "isOptional": true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1,
> "isUnique": true,
> "isIndexable": false
> }
> ],
> "superTypes": [],
> }
> {code}
> Some properties are missing after PUT update of attribute "isIndexable"



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message