apex-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From t..@apache.org
Subject [22/50] incubator-apex-core git commit: Fixed formatting and image
Date Wed, 02 Mar 2016 23:32:51 GMT
Fixed formatting and image


Project: http://git-wip-us.apache.org/repos/asf/incubator-apex-core/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-apex-core/commit/6b79095c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-apex-core/tree/6b79095c
Diff: http://git-wip-us.apache.org/repos/asf/incubator-apex-core/diff/6b79095c

Branch: refs/heads/APEXCORE-293
Commit: 6b79095cf6547da41bc55d7ea2cacc4e18899c17
Parents: d373226
Author: Chandni Singh <csingh@apache.org>
Authored: Wed Nov 4 18:53:00 2015 -0800
Committer: Thomas Weise <thomas@datatorrent.com>
Committed: Sun Feb 28 22:46:36 2016 -0800

----------------------------------------------------------------------
 autometrics/autometrics.md |  52 ++++++++++++++++++++++++----------------
 autometrics/images/adt.png | Bin 0 -> 169889 bytes
 2 files changed, 31 insertions(+), 21 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-apex-core/blob/6b79095c/autometrics/autometrics.md
----------------------------------------------------------------------
diff --git a/autometrics/autometrics.md b/autometrics/autometrics.md
index 8e7ffb2..364007f 100644
--- a/autometrics/autometrics.md
+++ b/autometrics/autometrics.md
@@ -51,12 +51,13 @@ For the `LineReceiver` operator, the application developer need not specify
any
 
 
 ## Building custom aggregators
-Platform cannot perform any meaningful aggregation for a non-numeric metric. In such case,
the operator or application developer can write custom aggregators. Let’s say, in the 
+Platform cannot perform any meaningful aggregation for a non-numeric metric. In such case,
the operator or application developer can write custom aggregators. Let’s say, if the `LineReceiver`
was modified to 
 
 
-Retrieving AutoMetrics
+# Retrieving AutoMetrics
 The Gateway REST API provides a way to retrieve the latest AutoMetrics for each logical operator.
 For example:
 
+```
 GET /ws/v2/applications/{appid}/logicalPlan/operators/{opName}
 {
     ...
@@ -65,20 +66,23 @@ GET /ws/v2/applications/{appid}/logicalPlan/operators/{opName}
     }
     ...
 }
+```
 
-System Metrics
+# System Metrics
 System metrics are standard operator metrics provided by the system.  Examples include:
 
-processed tuples per second
-emitted tuples per second
-total tuples processed
-total tuples emitted
-latency
-CPU percentage
-failure count
-checkpoint elapsed time
+- processed tuples per second
+- emitted tuples per second
+- total tuples processed
+- total tuples emitted
+- latency
+- CPU percentage
+- failure count
+- checkpoint elapsed time
 
 The Gateway REST API provides a way to retrieve the latest values for all of the above for
each of the logical operators in the application.
+
+```
 GET /ws/v2/applications/{appid}/logicalPlan/operators/{opName}
 {
     ...
@@ -91,17 +95,18 @@ GET /ws/v2/applications/{appid}/logicalPlan/operators/{opName}
     "tuplesProcessedPSMA": "{tuplesProcessedPSMA}", 
     ...
 }
+```
 
 However, just like AutoMetrics, the Gateway only provides the latest metrics.  For historical
metrics, we will need the help of App Data Tracker.
-App Data Tracker
 
+# App Data Tracker
 As discussed above, STRAM aggregates the AutoMetrics from physical operators (partitions)
to something that makes sense in one logical operator.  It pushes the aggregated AutoMetrics
values using Websocket to the Gateway at every second along with system metrics for each operator.
 Gateway relays the information to an application called App Data Tracker.  It is another
Apex application that runs in the background and further aggregates the incoming values by
time bucket and stores the values in HDHT.  It also allows the outside to retrieve the aggregated
AutoMetrics and system metrics through websocket interface.
 
-
-
+![AppDataTracker](images/adt.png)
 
 App Data Tracker is enabled by having these properties in dt-site.xml:
 
+```xml
 <property>
   <name>dt.appDataTracker.enable</name>
   <value>true</value>
@@ -114,19 +119,21 @@ App Data Tracker is enabled by having these properties in dt-site.xml:
   <name>dt.attr.METRICS_TRANSPORT</name>
   <value>builtin:AppDataTrackerFeed</value>
 </property>
+```
 
 All the applications launched after the App Data Tracker is enabled will have metrics sent
to it.
 
-Note: The App Data Tracker will be shown running in dtManage as a “system app”.  It will
show up if the “show system apps” button is pressed.
+**Note**: The App Data Tracker will be shown running in dtManage as a “system app”. 
It will show up if the “show system apps” button is pressed.
+
+By default, the time buckets App Data Tracker aggregates upon are one minute, one hour and
one day.  It can be overridden by changing the operator attribute `METRICS_DIMENSIONS_SCHEME`.
 
-By default, the time buckets App Data Tracker aggregates upon are one minute, one hour and
one day.  It can be overridden by changing the operator attribute METRICS_DIMENSIONS_SCHEME.
+Also by default, the app data tracker performs all these aggregations: SUM, MIN, MAX, AVG,
COUNT, FIRST, LAST on all number metrics.  You can also override by changing the same operator
attribute `METRICS_DIMENSIONS_SCHEME`, provided the custom aggregator is known to the App
Data Tracker.  (See next section)
 
-Also by default, the app data tracker performs all these aggregations: SUM, MIN, MAX, AVG,
COUNT, FIRST, LAST on all number metrics.  You can also override by changing the same operator
attribute METRICS_DIMENSIONS_SCHEME, provided the custom aggregator is known to the App Data
Tracker.  (See next section)
-Custom Aggregator in App Data Tracker
+# Custom Aggregator in App Data Tracker
 Custom aggregators allow you to do your own custom computation on statistics generated by
any of your applications. In order to implement a Custom aggregator you have to do two things:
 
-Combining new inputs with the current aggregation
-Combining two aggregations together into one aggregation
+1. Combining new inputs with the current aggregation
+2. Combining two aggregations together into one aggregation
 
 Let’s consider the case where we want to perform the following rolling average:
 
@@ -134,6 +141,7 @@ Y_n = ½ * X_n + ½ * X_n-1 + ¼ * X_n-2 + ⅛ * X_n-3 +...
 
 This aggregation could be performed by the following Custom Aggregator:
 
+```java
 @Name("IIRAVG")
 public class AggregatorIIRAVG extends AbstractIncrementalAggregator
 {
@@ -163,7 +171,9 @@ public class AggregatorIIRAVG extends AbstractIncrementalAggregator
     aggregateHelper(destAgg, srcAgg);
   }
 }
-Dashboards
+```
+
+# Dashboards
 With App Data Tracker enabled, you can visualize the AutoMetrics and system metrics in the
Dashboards within dtManage.   Refer back to the diagram in the App Data Tracker section, dtGateway
relays queries and query results to and from the App Data Tracker.  In this way, dtManage
sends queries and receives results from the App Data Tracker via dtGateway and uses the results
to let the user visualize the data.
 
 The Dashboards have some simple builtin widgets to visualize the data.  Line charts and bar
charts are some examples.

http://git-wip-us.apache.org/repos/asf/incubator-apex-core/blob/6b79095c/autometrics/images/adt.png
----------------------------------------------------------------------
diff --git a/autometrics/images/adt.png b/autometrics/images/adt.png
new file mode 100644
index 0000000..5b0d8ea
Binary files /dev/null and b/autometrics/images/adt.png differ


Mime
View raw message