flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chesnay Schepler <ches...@apache.org>
Subject Re: currentLowWatermark metric not reported for all tasks?
Date Fri, 30 Sep 2016 11:15:06 GMT
Hello Dan,

the technical reason is that this metric is only collected in the 
*InputProcessor classes, which aren't used for source tasks.

I do recall that there were discussions about source watermarks, but 
frankly i don't remember why we didn't add them.

In order to add them one would only have to modify the SourceContext 
classes to a) store the last emitted watermark and b) expose it through 
a metric.


On 29.09.2016 23:03, dan bress wrote:
> I am trying to measure the currentLowWatermark throughout my dataflow, but
> I am not seeing it for tasks with sources.  For those tasks I see these
> metrics:
> lastCheckpointSize
> numBytesInLocal
> numBytesInRemote
> numBytesOut
> why am I not seeing currentLowWatermark on these tasks?
> Thanks!
> Dan

View raw message