flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ufuk Celebi <...@apache.org>
Subject Re: record processing counter
Date Thu, 13 Nov 2014 15:14:36 GMT
Sure, go ahead. :)

I think following issues are related: FLINK-964, FLINK-766. There is also
an old GSoC proposal, which is related
https://github.com/stratosphere/stratosphere/wiki/GSoC-2014-Project-Proposal-Draft-by-Rajika-Kumarasiri
.

It would be cool if you could link the issues and also post the link.
<https://issues.apache.org/jira/browse/FLINK-964>

On Thu, Nov 13, 2014 at 4:00 PM, Stefano Bortoli <s.bortoli@gmail.com>
wrote:

> Hi Ufuk,
>
> thanks for the tip. I am already happy to know it will be on dashboard one
> day. Shall I open a ticket/feature request?
>
> For the moment I rely on old school note of execution time. The dataset
> will not change much, and I can have a reference once I get it to complete
> the first overall process. :-)
>
> saluti,
> Stefano
>
> 2014-11-13 15:46 GMT+01:00 Ufuk Celebi <uce@apache.org>:
>
>> Hey Stefano,
>>
>> this is definitely possible and also not hard to count. See the
>> RecordWriter class, which emits all records to the network runtime. The
>> problem at the moment is that we don't have a central service or the like,
>> which would be able to aggregate these kinds of runtime information and
>> then (for example) have them displayed on the web interface.
>>
>> This is on the agenda and in my opinion is very important.
>>
>> If this is important for you, I could add debug log messages or so, which
>> you would then have to parse on your own. Would that be feasible as a short
>> term solution?
>>
>> On Thu, Nov 13, 2014 at 3:15 PM, Stefano Bortoli <s.bortoli@gmail.com>
>> wrote:
>>
>>> Hi all,
>>>
>>> working with the dashboard I found extremely nice to have a monitor of
>>> the tasks processing. I wonder whether it is feasible, or could be
>>> interesting, to have a counter associated with the processing of each of
>>> the functions. I would not mind to have a little overhead in the overall
>>> process if I could understand where I am and be able to estimate the
>>> process duration. :-)
>>>
>>> what do you think?
>>>
>>> saluti,
>>> Stefano
>>>
>>
>>
>

Mime
View raw message