flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chesnay Schepler <ches...@apache.org>
Subject Re: Get EOF from PrometheusReporter in JM
Date Fri, 22 Sep 2017 14:07:40 GMT
The Prometheus reporter should work with 1.3.2.

Does this also occur with the reporter that currently exists in 1.4? (to 
rule out new bugs from the PR).

To investigate this further, please set the logging level to WARN and 
try again, as all errors in the metric system are logged on that level.

On 22.09.2017 10:33, Tony Wei wrote:
> Hi,
> I have built the Prometheus reporter package from this PR 
> https://github.com/apache/flink/pull/4586, and used it on Flink 1.3.2 
> to record every default metrics and those from `FlinkKafkaConsumer`.
> Originally, everything was fine. I could get those metrics in TM from 
> Prometheus just like I saw on Flink Web UI.
> However, when I turned to JM, I found Prometheus gives this error to 
> me: Get http://localhost:9249/metrics: EOF.
> I checked the log on JM and saw nothing in it. There was no error 
> message and 9249 port was still alive.
> To figure out what happened, I created another cluster and I found 
> Prometheus could connect to Flink cluster if there is no running job. 
> After JM triggered or completed the first checkpoint, Prometheus 
> started getting ERR_EMPTY_RESPONSE from JM, but not for TM. There was 
> still no error in log file and 9249 port was still alive.
> I was wondering where did the error occur. Flink or Prometheus reporter?
> Or It is incorrect to use Prometheus reporter on Flink 1.3.2 ? Thank you.
> Best Regards,
> Tony Wei

View raw message