atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shwetha GS <sshivalingamur...@hortonworks.com>
Subject Re: Review Request 53724: ATLAS-1116 Performance monitoring of backend methods in API requests
Date Wed, 16 Nov 2016 04:54:59 GMT


> On Nov. 15, 2016, 5:05 p.m., Apoorv Naik wrote:
> > server-api/pom.xml, line 65
> > <https://reviews.apache.org/r/53724/diff/1/?file=1563010#file1563010line65>
> >
> >     Why not use the existing MethodInterceptor mechanism that we're using for GraphTransaction
? Seems like we're overloading our project with lot of dependencies and compilation configurations.
The builds would get very finicky in my opinion.

Guice method interceptor will work only if the class is created through guice injection. Most
of the classes like GraphToTypedInstanceMapper, TypedInstanceToGraphMapper, GraphHelper which
need to be measured are not injected through guice. So, Guice method interceptor didn't work.
The code changes to create these classes through guice were hacky and cumbersome. Moreover,
guice method interceptor is done at runtime vs aspectj compile time injection


- Shwetha


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53724/#review155932
-----------------------------------------------------------


On Nov. 14, 2016, 11:06 a.m., Shwetha GS wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53724/
> -----------------------------------------------------------
> 
> (Updated Nov. 14, 2016, 11:06 a.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-1116
>     https://issues.apache.org/jira/browse/ATLAS-1116
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> Exposes annotations @Monitored and @Loggable on methods. 
> 
> @Monitored records how many times that method is called and total execution time in that
method. These metrics are aggregated at API level and are recorded in metric.log(one log line
per API call)
> @Loggable logs at the beginning of the method and ending of the method with arguments
and return values
> 
> These annotations are injected using aspectj plugin at compile time, there is no runtime
overhead except for the method invocation
> 
> Example metrics collected:
> entity submit - 2016-11-14 16:34:06,079  {GraphHelper.findVertex=[count=3, totalTimeMSec=60],
GraphHelper.setProperty=[count=15, totalTimeMSec=16], GraphHelper.createVertexWithoutIdentity=[count=1,
totalTimeMSec=2], TypedInstanceToGraphMapper.createVerticesAndDiscoverInstances=[count=1,
totalTimeMSec=62],
> TypedInstanceToGraphMapper.addOrUpdateAttributesAndTraits=[count=1, totalTimeMSec=10],
GraphHelper.getSingleValuedProperty=[count=33, totalTimeMSec=3], GraphHelper.getProperty=[count=3,
totalTimeMSec=0], GraphToTypedInstanceMapper.mapVertexToInstance=[count=3, totalTimeMSec=2],
GraphToTyped
> InstanceMapper.mapVertexToInstanceTraits=[count=3, totalTimeMSec=0], GraphToTypedInstanceMapper.mapGraphToTypedInstance=[count=3,
totalTimeMSec=6], FullTextMapper.mapRecursive=[count=1, totalTimeMSec=3], TypedInstanceToGraphMapper.mapTypedInstanceToGraph=[count=1,
totalTimeMSec=81], EntityR
> esource.submit=[count=1, totalTimeMSec=892]}
> 
> search - 2016-11-14 16:34:18,817  {GraphHelper.getSingleValuedProperty=[count=544, totalTimeMSec=20],
GraphHelper.getProperty=[count=47, totalTimeMSec=0], GraphHelper.getAdjacentEdgesByLabel=[count=18,
totalTimeMSec=3], GraphHelper.getEdgeForLabel=[count=18, totalTimeMSec=3], GraphToTypedInstanceMapper.mapVertexToInstance=[count=70,
totalTimeMSec=33], GraphToTypedInstanceMapper.mapVertexToInstanceTraits=[count=47, totalTimeMSec=5],
GraphToTypedInstanceMapper.mapGraphToTypedInstance=[count=47, totalTimeMSec=39], GraphHelper.getArrayElementsProperty=[count=10,
totalTimeMSec=0], MetadataDiscoveryResource.searchUsingQueryDSL=[count=1, totalTimeMSec=249],
MetadataDiscoveryResource.search=[count=1, totalTimeMSec=249]}
> 
> 
> Diffs
> -----
> 
>   distro/src/conf/atlas-log4j.xml 400cd3a 
>   pom.xml 1b3975f 
>   repository/pom.xml 949118b 
>   repository/src/main/java/org/apache/atlas/repository/graph/FullTextMapper.java 053e8ac

>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 7e47d30

>   repository/src/main/java/org/apache/atlas/repository/graph/GraphToTypedInstanceMapper.java
ceb6011 
>   repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
47ae5e1 
>   repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java
cb389a9 
>   server-api/pom.xml c183468 
>   server-api/src/main/aspect/org/apache/atlas/AtlasAspect.java PRE-CREATION 
>   server-api/src/main/aspect/org/apache/atlas/Loggable.java PRE-CREATION 
>   server-api/src/main/aspect/org/apache/atlas/Monitored.java PRE-CREATION 
>   server-api/src/main/java/org/apache/atlas/RequestContext.java ec38c11 
>   server-api/src/main/java/org/apache/atlas/metrics/Metrics.java PRE-CREATION 
>   server-api/src/test/aspect/org/apache/atlas/MonitoredAspectTest.java PRE-CREATION 
>   typesystem/src/main/resources/atlas-log4j.xml 8312657 
>   webapp/pom.xml 6d83cc0 
>   webapp/src/main/java/org/apache/atlas/web/filters/AuditFilter.java 79b5be4 
>   webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 4b45927 
>   webapp/src/main/java/org/apache/atlas/web/resources/DataSetLineageResource.java a11c0cf

>   webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java bbf01a6 
>   webapp/src/main/java/org/apache/atlas/web/resources/LineageResource.java 811c486 
>   webapp/src/main/java/org/apache/atlas/web/resources/MetadataDiscoveryResource.java
64344da 
>   webapp/src/main/java/org/apache/atlas/web/resources/TypesResource.java ace0d14 
> 
> Diff: https://reviews.apache.org/r/53724/diff/
> 
> 
> Testing
> -------
> 
> UT added and tested with atlas deployment
> 
> 
> Thanks,
> 
> Shwetha GS
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message