eagle-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (EAGLE-928) Refine system metric schema design and fix system metric collector
Date Tue, 28 Feb 2017 06:34:45 GMT

    [ https://issues.apache.org/jira/browse/EAGLE-928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15887398#comment-15887398
] 

ASF GitHub Bot commented on EAGLE-928:
--------------------------------------

Github user asfgit closed the pull request at:

    https://github.com/apache/eagle/pull/842


> Refine system metric schema design and fix system metric collector
> ------------------------------------------------------------------
>
>                 Key: EAGLE-928
>                 URL: https://issues.apache.org/jira/browse/EAGLE-928
>             Project: Eagle
>          Issue Type: Bug
>    Affects Versions: v0.5.0
>            Reporter: Hao Chen
>            Assignee: Hao Chen
>             Fix For: v0.5.0
>
>
> Refine system metric schema design and fix system metric collector
> h1. Principle
> Any metric (no mater sys metric) should at least have three fields: "metric", "group",
"timestamp", "value", 
> "group" is the metric category path separated with slash, for example "FIRST CATEGORY.SECOND
CATEGORY", which will be commonly used for well organizing metric groups in customized metric
dashboard.
> h1. Stream Name for System Metric
> {code}
> <stream>
>             <streamId>SYSTEM_METRIC_STREAM</streamId>
>             <description>System Metrics Stream including CPU, Network, Disk, etc.</description>
>             <columns>
>                 <column>
>                     <name>host</name>
>                     <type>string</type>
>                 </column>
>                 <column>
>                     <name>timestamp</name>
>                     <type>long</type>
>                 </column>
>                 <column>
>                     <name>metric</name>
>                     <type>string</type>
>                 </column>
>                 <column>
>                     <name>group</name>
>                     <type>string</type>
>                 </column>
>                 <column>
>                     <name>site</name>
>                     <type>string</type>
>                 </column>
>                 <column>
>                     <name>device</name>
>                     <type>string</type>
>                 </column>
>                 <column>
>                     <name>value</name>
>                     <type>double</type>
>                     <defaultValue>0.0</defaultValue>
>                 </column>
>             </columns>
>         </stream>
> {code}
> h1. Stream Schema for System metric
>     
>        * *metric*: [STRING] metric name string
>        * *group*: [STRING] metric group/type
>        * *timestamp*: [LONG] metric generation time
>        * *site*: [STRING] siteId
>        * *host*: [STRING] source host name
>        * *device*: [STRING] device name, like cpu
> h1. Sample CPU Metric
> {code}
> {
>     "timestamp": 1487918913569, 
>     "metric": "system.cpu.usage", 
>     "group": "system.cpu",
>     "site": "sandbox", 
>     "value": 0.058, 
>     "host": "sandbox.hortonworks.com", 
>     "device": "cpu7"
> }
> {code}
> h1. Sample Network Metrics
> {code}
> {
>     "timestamp": 1487918913569, 
>     "metric": "system.nic.transmitdrop", 
>     "group": "system.network",
>     "site": "sandbox", 
>     "value": 7724.0, 
>     "host": "sandbox.hortonworks.com", 
>     "device": "eth0"
> }
> {code}
> h1. Sample Metric Schema
> {code}
> {
>             "tags": {
>                 "site": "sandbox",
>                 "name": "system.memory.nfs_unstable.kb",
>                 "group": "system.memory"
>             },
>             "dimensionFields": [
>                 "host",
>                 "group",
>                 "site",
>                 "device"
>             ],
>             "metricFields": [
>                 "value"
>             ],
>             "granularity": "MINUTE",
>             "modifiedTimestamp": 1488190388479
>         }
> {code}
> h1. Stream Persist Topology
> {code}
>         environment.newApp(config)
>                 .fromStream("HADOOP_JMX_METRIC_STREAM")
>                 .saveAsMetric(
>                         MetricDescriptor.metricGroupAs((MetricGroupSelector) event ->
{
>                             if (event.containsKey("component")) {
>                                 return String.format("hadoop.%s", ((String) event.get("component")).toLowerCase());
>                             } else {
>                                 return "hadoop.metrics";
>                             }
>                         })
>                         .siteByField("site")
>                         .namedByField("metric")
>                         .eventTimeByField("timestamp")
>                         .dimensionFields("host", "component", "site")
>                         .granularity(Calendar.MINUTE)
>                         .valueField("value"))
>                 .fromStream("SYSTEM_METRIC_STREAM")
>                 .saveAsMetric(MetricDescriptor.metricGroupByField("group")
>                         .siteByField("site")
>                         .namedByField("metric")
>                         .eventTimeByField("timestamp")
>                         .dimensionFields("host", "group", "site", "device")
>                         .granularity(Calendar.MINUTE)
>                         .valueField("value")
>                 )
>                 .toTopology();
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message