flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ufuk Celebi <...@apache.org>
Subject Re: record processing counter
Date Thu, 13 Nov 2014 15:14:36 GMT
Sure, go ahead. :)

I think following issues are related: FLINK-964, FLINK-766. There is also
an old GSoC proposal, which is related

It would be cool if you could link the issues and also post the link.

On Thu, Nov 13, 2014 at 4:00 PM, Stefano Bortoli <s.bortoli@gmail.com>

> Hi Ufuk,
> thanks for the tip. I am already happy to know it will be on dashboard one
> day. Shall I open a ticket/feature request?
> For the moment I rely on old school note of execution time. The dataset
> will not change much, and I can have a reference once I get it to complete
> the first overall process. :-)
> saluti,
> Stefano
> 2014-11-13 15:46 GMT+01:00 Ufuk Celebi <uce@apache.org>:
>> Hey Stefano,
>> this is definitely possible and also not hard to count. See the
>> RecordWriter class, which emits all records to the network runtime. The
>> problem at the moment is that we don't have a central service or the like,
>> which would be able to aggregate these kinds of runtime information and
>> then (for example) have them displayed on the web interface.
>> This is on the agenda and in my opinion is very important.
>> If this is important for you, I could add debug log messages or so, which
>> you would then have to parse on your own. Would that be feasible as a short
>> term solution?
>> On Thu, Nov 13, 2014 at 3:15 PM, Stefano Bortoli <s.bortoli@gmail.com>
>> wrote:
>>> Hi all,
>>> working with the dashboard I found extremely nice to have a monitor of
>>> the tasks processing. I wonder whether it is feasible, or could be
>>> interesting, to have a counter associated with the processing of each of
>>> the functions. I would not mind to have a little overhead in the overall
>>> process if I could understand where I am and be able to estimate the
>>> process duration. :-)
>>> what do you think?
>>> saluti,
>>> Stefano

View raw message