flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chesnay Schepler <ches...@apache.org>
Subject Re: Monitoring backpressure
Date Wed, 07 Dec 2016 09:47:47 GMT
Hello Alan,

the backpressure information can be retrieved from the web ui's REST API 


This will give you a JSON object that looks something like this:


    |backpressure-level: "ok"|"low"|"high"|


            |subtask: 0|
            |backpressure-level: "ok"|"low"|"high"|
            |ratio: <ratio>|




For more details you can check out the JobVertexBackPressureHandler class.

On 07.12.2016 00:58, alan@opsclarity.com wrote:
> Hey Stephan,
> My company (OpsClarity) is building monitoring integration for flink, and
> being that backpressure is one of the most critical concepts in a streaming
> system, we need a way to expose backpressure state to a monitoring system
> (such as ours).  I see that the flink-ui has a way to sample the pipeline
> and mark a stage as high|med|ok wrt backpressure.  I'd love to be able to
> encode that as a metric, perhaps as simple as 2|1|0, so that we can plot
> backpressure state over time per stage.  This would also allow users to set
> alerts on backpressure state.
> How might we get access to this backpressure state information?
> Thanks,
> Alan
> --
> View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Monitoring-backpressure-tp9472p14868.html
> Sent from the Apache Flink Mailing List archive. mailing list archive at Nabble.com.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message