flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chesnay Schepler <ches...@apache.org>
Subject Re: Monitoring backpressure
Date Wed, 07 Dec 2016 09:47:47 GMT
Hello Alan,

the backpressure information can be retrieved from the web ui's REST API 
<https://ci.apache.org/projects/flink/flink-docs-release-1.2/monitoring/rest_api.html>.

|/jobs/<jobid>/vertices/<vertexid>/backpressure

This will give you a JSON object that looks something like this:

{
|

    |status:"ok"|"deprecated"|
    |backpressure-level: "ok"|"low"|"high"|
    |end-timestamp:<timestamp>|
    |subtasks:[|

        |{|

            |subtask: 0|
            |backpressure-level: "ok"|"low"|"high"|
            |ratio: <ratio>|

        |},|

    |}|
    ||

|}

For more details you can check out the JobVertexBackPressureHandler class.

Regards,
Chesnay
|
On 07.12.2016 00:58, alan@opsclarity.com wrote:
> Hey Stephan,
>
> My company (OpsClarity) is building monitoring integration for flink, and
> being that backpressure is one of the most critical concepts in a streaming
> system, we need a way to expose backpressure state to a monitoring system
> (such as ours).  I see that the flink-ui has a way to sample the pipeline
> and mark a stage as high|med|ok wrt backpressure.  I'd love to be able to
> encode that as a metric, perhaps as simple as 2|1|0, so that we can plot
> backpressure state over time per stage.  This would also allow users to set
> alerts on backpressure state.
>
> How might we get access to this backpressure state information?
>
> Thanks,
> Alan
>
>
>
> --
> View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Monitoring-backpressure-tp9472p14868.html
> Sent from the Apache Flink Mailing List archive. mailing list archive at Nabble.com.
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message