flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefano Bortoli <s.bort...@gmail.com>
Subject Re: record processing counter
Date Thu, 13 Nov 2014 15:00:42 GMT
Hi Ufuk,

thanks for the tip. I am already happy to know it will be on dashboard one
day. Shall I open a ticket/feature request?

For the moment I rely on old school note of execution time. The dataset
will not change much, and I can have a reference once I get it to complete
the first overall process. :-)


2014-11-13 15:46 GMT+01:00 Ufuk Celebi <uce@apache.org>:

> Hey Stefano,
> this is definitely possible and also not hard to count. See the
> RecordWriter class, which emits all records to the network runtime. The
> problem at the moment is that we don't have a central service or the like,
> which would be able to aggregate these kinds of runtime information and
> then (for example) have them displayed on the web interface.
> This is on the agenda and in my opinion is very important.
> If this is important for you, I could add debug log messages or so, which
> you would then have to parse on your own. Would that be feasible as a short
> term solution?
> On Thu, Nov 13, 2014 at 3:15 PM, Stefano Bortoli <s.bortoli@gmail.com>
> wrote:
>> Hi all,
>> working with the dashboard I found extremely nice to have a monitor of
>> the tasks processing. I wonder whether it is feasible, or could be
>> interesting, to have a counter associated with the processing of each of
>> the functions. I would not mind to have a little overhead in the overall
>> process if I could understand where I am and be able to estimate the
>> process duration. :-)
>> what do you think?
>> saluti,
>> Stefano

View raw message