Mailing-List: contact dev-help@atlas.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@atlas.incubator.apache.org
Content-Type: multipart/alternative;
 boundary="===============4480171748694175357=="
MIME-Version: 1.0
Subject: Re: Review Request 53724: ATLAS-1116 Performance monitoring of backend
 methods in API requests
From: Shwetha GS <sshivalingamurthy@hortonworks.com>
To: Vimal Sharma <visharma@hortonworks.com>,
 Apoorv Naik <naik.apoorv@gmail.com>, atlas <dev@atlas.incubator.apache.org>,
 Shwetha GS <sshivalingamurthy@hortonworks.com>
Date: Fri, 18 Nov 2016 10:45:39 -0000
Message-ID: <20161118104539.20485.80673@reviews.apache.org>
Auto-Submitted: auto-generated
Sender: Shwetha GS <noreply@reviews.apache.org>
References: <20161116093323.20485.79406@reviews.apache.org>
In-Reply-To: <20161116093323.20485.79406@reviews.apache.org>
Reply-To: Shwetha GS <sshivalingamurthy@hortonworks.com>
archived-at: Fri, 18 Nov 2016 10:45:52 -0000

--===============4480171748694175357==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53724/
-----------------------------------------------------------

(Updated Nov. 18, 2016, 10:45 a.m.)


Review request for atlas.


Changes
-------

Aspectj plugin disabled by default. Added instructions on how to enable instrumentation at compile time and to disable metrics at runtime when atlas is already instrumented.


Bugs: ATLAS-1116
    https://issues.apache.org/jira/browse/ATLAS-1116


Repository: atlas


Description
-------

Exposes annotations @Monitored and @Loggable on methods. 

@Monitored records how many times that method is called and total execution time in that method. These metrics are aggregated at API level and are recorded in metric.log(one log line per API call)
@Loggable logs at the beginning of the method and ending of the method with arguments and return values

These annotations are injected using aspectj plugin at compile time, there is no runtime overhead except for the method invocation

Example metrics collected:
entity submit - 2016-11-14 16:34:06,079  {GraphHelper.findVertex=[count=3, totalTimeMSec=60], GraphHelper.setProperty=[count=15, totalTimeMSec=16], GraphHelper.createVertexWithoutIdentity=[count=1, totalTimeMSec=2], TypedInstanceToGraphMapper.createVerticesAndDiscoverInstances=[count=1, totalTimeMSec=62],
TypedInstanceToGraphMapper.addOrUpdateAttributesAndTraits=[count=1, totalTimeMSec=10], GraphHelper.getSingleValuedProperty=[count=33, totalTimeMSec=3], GraphHelper.getProperty=[count=3, totalTimeMSec=0], GraphToTypedInstanceMapper.mapVertexToInstance=[count=3, totalTimeMSec=2], GraphToTyped
InstanceMapper.mapVertexToInstanceTraits=[count=3, totalTimeMSec=0], GraphToTypedInstanceMapper.mapGraphToTypedInstance=[count=3, totalTimeMSec=6], FullTextMapper.mapRecursive=[count=1, totalTimeMSec=3], TypedInstanceToGraphMapper.mapTypedInstanceToGraph=[count=1, totalTimeMSec=81], EntityR
esource.submit=[count=1, totalTimeMSec=892]}

search - 2016-11-14 16:34:18,817  {GraphHelper.getSingleValuedProperty=[count=544, totalTimeMSec=20], GraphHelper.getProperty=[count=47, totalTimeMSec=0], GraphHelper.getAdjacentEdgesByLabel=[count=18, totalTimeMSec=3], GraphHelper.getEdgeForLabel=[count=18, totalTimeMSec=3], GraphToTypedInstanceMapper.mapVertexToInstance=[count=70, totalTimeMSec=33], GraphToTypedInstanceMapper.mapVertexToInstanceTraits=[count=47, totalTimeMSec=5], GraphToTypedInstanceMapper.mapGraphToTypedInstance=[count=47, totalTimeMSec=39], GraphHelper.getArrayElementsProperty=[count=10, totalTimeMSec=0], MetadataDiscoveryResource.searchUsingQueryDSL=[count=1, totalTimeMSec=249], MetadataDiscoveryResource.search=[count=1, totalTimeMSec=249]}


Diffs (updated)
-----

  distro/src/conf/atlas-log4j.xml 400cd3a 
  docs/src/site/twiki/Configuration.twiki 7f24f88 
  docs/src/site/twiki/InstallationSteps.twiki 4dae7a3 
  pom.xml 1b3975f 
  repository/pom.xml 949118b 
  repository/src/main/java/org/apache/atlas/repository/graph/FullTextMapper.java 053e8ac 
  repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 7e47d30 
  repository/src/main/java/org/apache/atlas/repository/graph/GraphToTypedInstanceMapper.java ceb6011 
  repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java 47ae5e1 
  repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java cb389a9 
  server-api/pom.xml c183468 
  server-api/src/main/java/org/apache/atlas/RequestContext.java ec38c11 
  server-api/src/main/java/org/apache/atlas/aspect/AtlasAspect.java PRE-CREATION 
  server-api/src/main/java/org/apache/atlas/aspect/Loggable.java PRE-CREATION 
  server-api/src/main/java/org/apache/atlas/aspect/Monitored.java PRE-CREATION 
  server-api/src/main/java/org/apache/atlas/metrics/Metrics.java PRE-CREATION 
  server-api/src/test/aspect/org/apache/atlas/MonitoredAspectTest.java PRE-CREATION 
  typesystem/src/main/resources/atlas-log4j.xml 8312657 
  webapp/pom.xml 2bef274 
  webapp/src/main/java/org/apache/atlas/web/filters/AuditFilter.java 79b5be4 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 4b45927 
  webapp/src/main/java/org/apache/atlas/web/resources/DataSetLineageResource.java a11c0cf 
  webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 230265b 
  webapp/src/main/java/org/apache/atlas/web/resources/LineageResource.java 811c486 
  webapp/src/main/java/org/apache/atlas/web/resources/MetadataDiscoveryResource.java ac89869 
  webapp/src/main/java/org/apache/atlas/web/resources/TypesResource.java e899fcf 

Diff: https://reviews.apache.org/r/53724/diff/


Testing
-------

UT added and tested with atlas deployment


Thanks,

Shwetha GS


--===============4480171748694175357==--