flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Dimiduk <ndimi...@gmail.com>
Subject Re: Tiny topology shows '0' for all stats.
Date Tue, 15 Dec 2015 17:25:21 GMT
For my own understanding, are you suggesting the FLINK-2944 (or a subtask)
is the appropriate place to implement exposure of metrics such as bytes,
records in, out of Streaming sources and sinks?

On Tue, Dec 15, 2015 at 5:24 AM, Niels Basjes <Niels@basjes.nl> wrote:

> Hi,
>
> @Ufuk: I added the env.disableOperatorChaining() and indeed now I see two
> things on the screen and there are numbers counting what has happened.
> @Stephan: Yes, I understand these numbers now.
>
> I found that this is already a jira ticket to add what I was looking for:
> https://issues.apache.org/jira/browse/FLINK-2944
> https://issues.apache.org/jira/browse/FLINK-3130
>
> Niels
>
>
>
> On Mon, Dec 14, 2015 at 5:03 PM, Ufuk Celebi <uce@apache.org> wrote:
>
>>
>> > On 14 Dec 2015, at 16:25, Niels Basjes <Niels@basjes.nl> wrote:
>> >
>> > Hi,
>> >
>> > I have a very small topology here.
>> > In fact this is a thing that generates synthetic data and puts it into
>> Kafka.
>> > When looking at the web UI I see that all counters (i.e. Bytes
>> received, Records received, Bytes sent,  Records sent) all remain 0.
>> > I verified and I'm seeing thousands of records arriving into Kafka.
>> >
>> > Is this a bug in Flink or am I misinterpreting the meaning of these
>> numbers?
>>
>> Sources and sinks do not show the number of received or sent records,
>> because of the internals of how these numbers are collected. I agree that
>> this is confusing. Big +1 to improve this.
>>
>> You actually don’t see any numbers, because the operators are chained and
>> hence you only have one task, which acts as both source and sink.
>>
>> You should see some sent and received numbers if you break up the chain
>> (env.disableOperatorChaining()). Can you confirm this?
>>
>> – Ufuk
>>
>>
>
>
> --
> Best regards / Met vriendelijke groeten,
>
> Niels Basjes
>

Mime
View raw message