flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hai Zhou_UTC+8 (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-7608) LatencyGauge change to histogram metric
Date Wed, 20 Sep 2017 13:14:01 GMT

    [ https://issues.apache.org/jira/browse/FLINK-7608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173155#comment-16173155
] 

Hai Zhou_UTC+8 commented on FLINK-7608:
---------------------------------------

BTW, we collect all the job metrics in production is such as: **implement a kafkareporter
 -->  kafka2opentsdb (a flink job)  -->  Grafana**.

custom the scope task format like:
{noformat}
metrics.scope.task:  TASK.<host>.<job_name>.<task_name>.<subtask_index>
{noformat}

parse the metric into the opentsdb data format like:
{noformat}
    {
        "metric":"numBytesInLocalPerSecond",
        "tags":{
            "subtask_index":"0",
            "host":"xxx-xxx-xx1588",
            "job_name":"xxxx_xxxx_xxxx",
            "task_name":"Source--xxxx"
        },
        "timestamp":1505390758,
        "value":2342
    }
{noformat}


> LatencyGauge change to  histogram metric
> ----------------------------------------
>
>                 Key: FLINK-7608
>                 URL: https://issues.apache.org/jira/browse/FLINK-7608
>             Project: Flink
>          Issue Type: Bug
>          Components: Metrics
>            Reporter: Hai Zhou_UTC+8
>            Assignee: Hai Zhou_UTC+8
>            Priority: Blocker
>             Fix For: 1.4.0, 1.3.3
>
>
> I used slf4jReporter[https://issues.apache.org/jira/browse/FLINK-4831]  to export metrics
the log file.
> I found:
> {noformat}
> -- Gauges ---------------------------------------------------------------------
> ......
> zhouhai-mbp.taskmanager.f3fd3a269c8c3da4e8319c8f6a201a57.Flink Streaming Job.Map.0.latency:
>  value={LatencySourceDescriptor{vertexID=1, subtaskIndex=-1}={p99=116.0, p50=59.5, min=11.0,
max=116.0, p95=116.0, mean=61.833333333333336}}
> zhouhai-mbp.taskmanager.f3fd3a269c8c3da4e8319c8f6a201a57.Flink Streaming Job.Sink- Unnamed.0.latency:

> value={LatencySourceDescriptor{vertexID=1, subtaskIndex=0}={p99=195.0, p50=163.5, min=115.0,
max=195.0, p95=195.0, mean=161.0}}
> ......
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message