flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Deirdre Kong <deirdre.y.k...@gmail.com>
Subject Re: Configuring Ports for Job/Task Manager Metrics
Date Fri, 31 Aug 2018 05:26:32 GMT
Hi Vino/Chesnay,

Thank you for the info.

I am actually using Yarn for deployment.  Flink is installed in AWS EMR, so
sometimes jm and tm processes are deployed in the same container, sometimes
they are deployed in different containers.  I would need to configure
Prometheus to listen on a specify port for  JM and TM reporter.  So even if
JM is bounced, I can get the JM metrics on the same port each time.

Can you elaborate what do you mean by specify a separate Flink installation
package for it?

Chesnay, do you have any insights on this?

Thanks,
Deirdre

On Thu, Aug 30, 2018 at 7:18 PM vino yang <yanghua1127@gmail.com> wrote:

> Hi Deirdre,
>
> If you run multiple Flink component (jm/tm) processes on one physical
> node, it is recommended to specify the port range to avoid conflicts[1], I
> guess this is based on the same Flink binary installation package.
> If you want to always have the TM reporter running on the same port, you
> can specify a separate Flink installation package for it, explicitly
> specifying this port in the configuration file for this installation
> package.
> However, you still need to pay attention to port conflicts with other
> components.
>
> The issue you provided is handled by Chesnay, so maybe Chesnay opinion is
> more appropriate. Ping Chesnay for you.
>
> [1]:
> https://ci.apache.org/projects/flink/flink-docs-release-1.6/monitoring/metrics.html#prometheus-orgapacheflinkmetricsprometheusprometheusreporter
>
> Thanks, vino.
>
> Deirdre Kong <deirdre.y.kong@gmail.com> 于2018年8月31日周五 上午5:25写道:
>
>> Hi,
>>
>> I am running Flink in Amazon EMR.  In flink-conf.yaml, I have `metrics.reporter.prom.port:
>> 9249-9250`
>> Depending whether the job manager and task manager are running in the
>> same node, the task manager metrics are reported on port 9250 (if running
>> on same node as job manager), or on port 9249 (if running on a different
>> node).
>>
>> Is there a way to configure so that the task manager metrics are always
>> reported on port 9250?
>> I saw a post <https://issues.apache.org/jira/browse/FLINK-8284> that we
>> can "provide each *Manager with a separate configuration."  How to do that?
>>
>> Thanks
>>
>

Mime
View raw message