flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chesnay Schepler <ches...@apache.org>
Subject Re: Task and Operator Metrics in Flink 1.3
Date Tue, 13 Jun 2017 08:26:08 GMT
The scopes look OK to me.

Let's try to narrow down the problem areas a bit:

 1. Did this work with the same setup before 1.3?
 2. Are all task/operator metrics available in the metrics tab of the
    dashboard?
 3. Are there any warnings in the TaskManager logs from the
    MetricRegistry or StatsDReporter?

My *guess *would be that the operator/task metrics contain characters 
that either StatsD or telegraf don't allow,
which causes them to be dropped.

On 12.06.2017 20:32, Dail, Christopher wrote:
>
> I’m using the Flink 1.3.0 release and am not seeing all of the metrics 
> that I would expect to see. I have flink configured to write out 
> metrics via statsd and I am consuming this with telegraf. Initially I 
> thought this was an issue with telegraf parsing the data generated. I 
> dumped all of the metrics going into telegraf using tcpdump and found 
> that there was a bunch of data missing that I expect.
>
> I’m using this as a reference for what metrics I expect:
>
> https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/metrics.html
>
> I see all of the JobManager and TaskManager level metrics. Things like 
> Status.JVM.* are coming through. TaskManager Status.Network are there 
> (but not Task level buffers). The ‘Cluster’ metrics are there.
>
> This IO section contains task and operator level metrics (like what is 
> available on the dashboard). I’m not seeing any of these metrics 
> coming through when using statsd.
>
> I’m configuring flink with this configuration:
>
> metrics.reporters: statsd
>
> metrics.reporter.statsd.class: 
> org.apache.flink.metrics.statsd.StatsDReporter
>
> metrics.reporter.statsd.host: hostname
>
> metrics.reporter.statsd.port: 8125
>
> # Customized Scopes
>
> metrics.scope.jm: flink.jm
>
> metrics.scope.jm.job: flink.jm.<job_name>
>
> metrics.scope.tm: flink.tm.<tm_id>
>
> metrics.scope.tm.job: flink.tm.<tm_id>.<job_name>
>
> metrics.scope.task: 
> flink.tm.<tm_id>.<job_name>.<task_name>.<subtask_index>
>
> metrics.scope.operator: 
> flink.tm.<tm_id>.<job_name>.<operator_name>.<subtask_index>
>
> I have tried with and without specifically setting the metrics.scope 
> values.
>
> Is anyone else having similar issues with metrics in 1.3?
>
> Thanks
>
> *Chris Dail*
>
> Director, Software Engineering
>
> *Dell EMC* | Infrastructure Solutions Group
>
> mobile +1 506 863 4675
>
> Christopher.Dail@dell.com <mailto:Christopher.Dail@dell.com>
>


Mime
View raw message