flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: currentLowWatermark metric not reported for all tasks?
Date Fri, 30 Sep 2016 12:48:41 GMT
I think what you describe, Chesnay, is exactly what we should do...

On Fri, Sep 30, 2016 at 1:15 PM, Chesnay Schepler <chesnay@apache.org>
wrote:

> Hello Dan,
>
> the technical reason is that this metric is only collected in the
> *InputProcessor classes, which aren't used for source tasks.
>
> I do recall that there were discussions about source watermarks, but
> frankly i don't remember why we didn't add them.
>
> In order to add them one would only have to modify the SourceContext
> classes to a) store the last emitted watermark and b) expose it through a
> metric.
>
> Regards,
> Chesnay
>
>
> On 29.09.2016 23:03, dan bress wrote:
>
>> I am trying to measure the currentLowWatermark throughout my dataflow, but
>> I am not seeing it for tasks with sources.  For those tasks I see these
>> metrics:
>>
>> lastCheckpointSize
>> numBytesInLocal
>> numBytesInRemote
>> numBytesOut
>>
>> why am I not seeing currentLowWatermark on these tasks?
>>
>> Thanks!
>>
>> Dan
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message