atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Srikanth Venkat (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ATLAS-1114) Performance improvements for create/update entities
Date Thu, 26 Jan 2017 17:25:24 GMT

    [ https://issues.apache.org/jira/browse/ATLAS-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15840088#comment-15840088
] 

Srikanth Venkat commented on ATLAS-1114:
----------------------------------------

Jeff,

It would be useful if along with the patch you can provide test cases or scenarios that were
used to identify this issue and can be used to baseline and can be used to validate/benchmark
improvements in these areas moving forward.

> Performance improvements for create/update entities
> ---------------------------------------------------
>
>                 Key: ATLAS-1114
>                 URL: https://issues.apache.org/jira/browse/ATLAS-1114
>             Project: Atlas
>          Issue Type: Bug
>            Reporter: Jeffrey Hagelberg
>            Assignee: Jeffrey Hagelberg
>         Attachments: rb51092.patch
>
>
> General Atlas Perforance Improvements
> Apply performance fixes for create/update entities from IBM fork to Atlas.  During our
performance profiling, we found a number of performance hotspots in JProfiler.  Our main findings
were
>    - multiple queries were being executed for each instance being created/updated to
find matches by unique attribute.  
>   - one query was being executed for each instance being created/updated to find the
corresponding vertex if there is one
>   - Calculating the value of the full text property was taking a significant portion
of the time to create/update entities, mainly due to its calls to getVertexForGUID
> The changes we put in do the following:
> - batch lookups by guid when create/update entities.  Execute one AtlasGraphQuery to
find them all.
> - batch lookups by unique attribute when create/update entities.  Execute one AtlasGraphQuery
per class to find unique attribute matches.
> - find all existing vertices up front during create/update entity.  Use those vertices
during the graph mapping process to avoid running unnecessary graph queries
> - reuse reference vertices from instance to graph mapping when computing full text property



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message