atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shwetha GS <sshivalingamur...@hortonworks.com>
Subject Re: Review Request 53724: ATLAS-1116 Performance monitoring of backend methods in API requests
Date Fri, 18 Nov 2016 10:45:39 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53724/
-----------------------------------------------------------

(Updated Nov. 18, 2016, 10:45 a.m.)


Review request for atlas.


Changes
-------

Aspectj plugin disabled by default. Added instructions on how to enable instrumentation at
compile time and to disable metrics at runtime when atlas is already instrumented.


Bugs: ATLAS-1116
    https://issues.apache.org/jira/browse/ATLAS-1116


Repository: atlas


Description
-------

Exposes annotations @Monitored and @Loggable on methods. 

@Monitored records how many times that method is called and total execution time in that method.
These metrics are aggregated at API level and are recorded in metric.log(one log line per
API call)
@Loggable logs at the beginning of the method and ending of the method with arguments and
return values

These annotations are injected using aspectj plugin at compile time, there is no runtime overhead
except for the method invocation

Example metrics collected:
entity submit - 2016-11-14 16:34:06,079  {GraphHelper.findVertex=[count=3, totalTimeMSec=60],
GraphHelper.setProperty=[count=15, totalTimeMSec=16], GraphHelper.createVertexWithoutIdentity=[count=1,
totalTimeMSec=2], TypedInstanceToGraphMapper.createVerticesAndDiscoverInstances=[count=1,
totalTimeMSec=62],
TypedInstanceToGraphMapper.addOrUpdateAttributesAndTraits=[count=1, totalTimeMSec=10], GraphHelper.getSingleValuedProperty=[count=33,
totalTimeMSec=3], GraphHelper.getProperty=[count=3, totalTimeMSec=0], GraphToTypedInstanceMapper.mapVertexToInstance=[count=3,
totalTimeMSec=2], GraphToTyped
InstanceMapper.mapVertexToInstanceTraits=[count=3, totalTimeMSec=0], GraphToTypedInstanceMapper.mapGraphToTypedInstance=[count=3,
totalTimeMSec=6], FullTextMapper.mapRecursive=[count=1, totalTimeMSec=3], TypedInstanceToGraphMapper.mapTypedInstanceToGraph=[count=1,
totalTimeMSec=81], EntityR
esource.submit=[count=1, totalTimeMSec=892]}

search - 2016-11-14 16:34:18,817  {GraphHelper.getSingleValuedProperty=[count=544, totalTimeMSec=20],
GraphHelper.getProperty=[count=47, totalTimeMSec=0], GraphHelper.getAdjacentEdgesByLabel=[count=18,
totalTimeMSec=3], GraphHelper.getEdgeForLabel=[count=18, totalTimeMSec=3], GraphToTypedInstanceMapper.mapVertexToInstance=[count=70,
totalTimeMSec=33], GraphToTypedInstanceMapper.mapVertexToInstanceTraits=[count=47, totalTimeMSec=5],
GraphToTypedInstanceMapper.mapGraphToTypedInstance=[count=47, totalTimeMSec=39], GraphHelper.getArrayElementsProperty=[count=10,
totalTimeMSec=0], MetadataDiscoveryResource.searchUsingQueryDSL=[count=1, totalTimeMSec=249],
MetadataDiscoveryResource.search=[count=1, totalTimeMSec=249]}


Diffs (updated)
-----

  distro/src/conf/atlas-log4j.xml 400cd3a 
  docs/src/site/twiki/Configuration.twiki 7f24f88 
  docs/src/site/twiki/InstallationSteps.twiki 4dae7a3 
  pom.xml 1b3975f 
  repository/pom.xml 949118b 
  repository/src/main/java/org/apache/atlas/repository/graph/FullTextMapper.java 053e8ac 
  repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 7e47d30 
  repository/src/main/java/org/apache/atlas/repository/graph/GraphToTypedInstanceMapper.java
ceb6011 
  repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
47ae5e1 
  repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java
cb389a9 
  server-api/pom.xml c183468 
  server-api/src/main/java/org/apache/atlas/RequestContext.java ec38c11 
  server-api/src/main/java/org/apache/atlas/aspect/AtlasAspect.java PRE-CREATION 
  server-api/src/main/java/org/apache/atlas/aspect/Loggable.java PRE-CREATION 
  server-api/src/main/java/org/apache/atlas/aspect/Monitored.java PRE-CREATION 
  server-api/src/main/java/org/apache/atlas/metrics/Metrics.java PRE-CREATION 
  server-api/src/test/aspect/org/apache/atlas/MonitoredAspectTest.java PRE-CREATION 
  typesystem/src/main/resources/atlas-log4j.xml 8312657 
  webapp/pom.xml 2bef274 
  webapp/src/main/java/org/apache/atlas/web/filters/AuditFilter.java 79b5be4 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 4b45927 
  webapp/src/main/java/org/apache/atlas/web/resources/DataSetLineageResource.java a11c0cf

  webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 230265b 
  webapp/src/main/java/org/apache/atlas/web/resources/LineageResource.java 811c486 
  webapp/src/main/java/org/apache/atlas/web/resources/MetadataDiscoveryResource.java ac89869

  webapp/src/main/java/org/apache/atlas/web/resources/TypesResource.java e899fcf 

Diff: https://reviews.apache.org/r/53724/diff/


Testing
-------

UT added and tested with atlas deployment


Thanks,

Shwetha GS


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message