flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bastien dine <bastien.d...@gmail.com>
Subject Re: Flink - Metric are not reported
Date Wed, 28 Nov 2018 15:01:26 GMT
Yea, that was I was thinking..
Batch can be quick
Can I report metric on "added" action ? (i should override the
notifyOnAddedMetric to report ?)

------------------

Bastien DINE
Data Architect / Software Engineer / Sysadmin
bastiendine.io


Le mer. 28 nov. 2018 à 15:54, Chesnay Schepler <chesnay@apache.org> a
écrit :

> How quick does job batch terminate? Metrics are unregistered once the job
> ends; if the job duration is shorter than the report interval they may
> never be exposed.
>
> On 28.11.2018 15:18, bastien dine wrote:
>
> Hello Chesnay,
>
> Thanks for your response !
> I have logs enable (info), slf4jReporter is working, I can see :
>
> 15:16:00.112 [Flink-MetricRegistry-thread-1] INFO
> org.apache.flink.metrics.slf4j.Slf4jReporter  -
> =========================== Starting metrics report
> ===========================
>
> -- Counters
> -------------------------------------------------------------------
>
> -- Gauges
> ---------------------------------------------------------------------
> ....
>
> On both jobmanager & taskmanager,
> *BUT* i see only system metrics, not my custom one..
> Am i missing something in declaration in my topology ?
>
> *Note* : I am using DataSet API (so my program is batch, and not
> continuous)
>
> Regards,
> Bastien
>
> ------------------
>
> Bastien DINE
> Data Architect / Software Engineer / Sysadmin
> bastiendine.io
>
>
> Le mar. 27 nov. 2018 à 17:07, Chesnay Schepler <chesnay@apache.org> a
> écrit :
>
>> Please enable WARN logging and check for warnings by the SLF4JReporter
>> and/or MetricQueryService.
>>
>> On 27.11.2018 17:00, bastien dine wrote:
>>
>> Hello everyone,
>>
>> Once again I require your help !
>> I am trying to report custom metric (see my code below)
>> Yet, I do not see them anywhere.. nor in the metric tab from my tasks,
>> nor in the rest API, nor in the declared slf4j reporter..
>> Can someone help me to debug this ..
>>
>> Here is my RichMap function :
>>
>> public class MetricGaugeRichMap<T, E extends MetricTuple<T>> extends
RichMapFunction<E, E> {
>>
>>     private transient T valueToExpose;
>>     private final String metricGroup;
>>     private final String metricName;
>>
>>
>>     public MetricGaugeRichMap(String metricGroup, String metricName) {
>>         this.metricGroup = metricGroup;
>>         this.metricName = metricName;
>>     }
>>
>>     @Override    public void open(Configuration config) {
>>         getRuntimeContext()
>>                 .getMetricGroup()
>>                 .addGroup(metricGroup)
>>                 .gauge(metricName, (Gauge<T>) () -> valueToExpose);
>>     }
>>
>>     @Override    public E map(E metricTuple) throws Exception {
>>         valueToExpose = metricTuple.getMetricValue();
>>         return metricTuple;
>>     }
>> }
>>
>> calling from topology :
>>
>> env.fromElements(new MetricTuple<>(metricGroup, metricName, metricValue))
>>     .map(new MetricGaugeRichMap<>(metricGroup, metricName))
>>     .output(new MetricGaugeOutputFormat<>()); // dummy output
>>
>> ------------------
>>
>>
>> Bastien DINE
>> Data Architect / Software Engineer / Sysadmin
>> bastiendine.io
>>
>>
>>
>

Mime
View raw message