atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Madhan Neethiraj <mad...@apache.org>
Subject Re: Review Request 55813: Porting performance and stability changes made in 0.7 branch into master
Date Mon, 23 Jan 2017 22:59:59 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55813/#review162725
-----------------------------------------------------------


Fix it, then Ship it!





repository/src/main/java/org/apache/atlas/discovery/graph/GraphBackedDiscoveryService.java
(line 127)
<https://reviews.apache.org/r/55813/#comment234051>

    Please change log level to 'debug'.



repository/src/main/java/org/apache/atlas/discovery/graph/GraphBackedDiscoveryService.java
(line 153)
<https://reviews.apache.org/r/55813/#comment234052>

    Please change log level to 'debug'.


- Madhan Neethiraj


On Jan. 23, 2017, 10:09 p.m., Sarath Subramanian wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55813/
> -----------------------------------------------------------
> 
> (Updated Jan. 23, 2017, 10:09 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj and Suma Shivaprasad.
> 
> 
> Bugs: ATLAS-1403
>     https://issues.apache.org/jira/browse/ATLAS-1403
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> Currently DSL uses a fill function during Gremlin Translation to merge results by typeName
and superTypeName and fill function loads the resulting vertices in memory. This causes significant
memory usage and ATLAS server spends lot of time doing GC instead of useful work resulting
in OOO sometimes ( when GC is not able to recover and search queries are run in parallel)
> The proposal is to replace this with typeName checks along by finding all the subtypes
for a given type and using an IN clause in the filter.
> For eg:
> Query = Person where (birthday < "1950-01-01T02:35:58.440Z") limit 40 offset 0
> Optimized query
> Gremlin Query = L:
> {g.V.has("__typeName", T.in, ['Person','Manager']).and(_().has("Person.birthday", T.lt,
-631142641560)) [0..<40].toList()}
> 
> 
> Diffs
> -----
> 
>   repository/src/main/java/org/apache/atlas/discovery/DataSetLineageService.java fd5dba7

>   repository/src/main/java/org/apache/atlas/discovery/graph/DefaultGraphPersistenceStrategy.java
266f27c 
>   repository/src/main/java/org/apache/atlas/discovery/graph/GraphBackedDiscoveryService.java
b637f90 
>   repository/src/main/java/org/apache/atlas/gremlin/Gremlin2ExpressionFactory.java 41dc65f

>   repository/src/main/java/org/apache/atlas/gremlin/GremlinExpressionFactory.java 3677544

>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 889236c

>   repository/src/main/scala/org/apache/atlas/query/ClosureQuery.scala daef582 
>   repository/src/main/scala/org/apache/atlas/query/GraphPersistenceStrategies.scala a9dcdff

>   repository/src/main/scala/org/apache/atlas/query/GremlinEvaluator.scala ade4176 
>   repository/src/test/scala/org/apache/atlas/query/GremlinTest2.scala 33513c5 
> 
> Diff: https://reviews.apache.org/r/55813/diff/
> 
> 
> Testing
> -------
> 
> Ran Unit Tests and was successful.
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message