flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wouter Zorgdrager <W.D.Zorgdra...@tudelft.nl>
Subject Flink and Prometheus setup in K8s
Date Mon, 13 May 2019 12:16:34 GMT
Hey all,

I'm working on a deployment setup with Flink and Prometheus on Kubernetes.
I'm running into the following issues:

1) Is it possible to use the default Flink Docker image [1] and enable the
Prometheus reporter? Modifying the flink-config.yaml is easy, but somehow
the Prometheus reporter jar needs to be moved within the image. This is
easy if use my own Dockerfile (as done here [2]) , but I prefer using the
official one.
2) I can define the jobmanager/taskmanager metric endpoints statically, but
w.r.t. scaling I prefer to have these resolved/discovered dynamically. Did
anyone get a working setup on this? I came across this resource for YARN
[3], is there something similar for Kubernetes? Or are there any other ways
of configuring Prometheus to pick this up automatically?

Thanks a lot for your help!

Kind regards,
Wouter

[1]: https://hub.docker.com/_/flink/
[2]:
https://github.com/mbode/flink-prometheus-example/blob/master/Dockerfile
[3]: https://github.com/eastcirclek/flink-service-discovery

Mime
View raw message