flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrea Spina <andrea.sp...@radicalbit.io>
Subject Re: Flink sent/received bytes related metrics: how are they calculated?
Date Fri, 14 Jun 2019 11:58:14 GMT
Hi Chesnay, just one downstream: Sin (Source: Enriched Code) outcome is the
right part of the following operator as in the figure; this operator is the
exclusive downstream of Sin.
Thanks,

[image: Screenshot 2019-06-14 at 13.52.52.png]

Il giorno ven 14 giu 2019 alle ore 12:23 Chesnay Schepler <
chesnay@apache.org> ha scritto:

> How does the *P1 *pipeline look like? Are there 2 downstream operators
> reading from *Sin* (in this case the number of bytes would be measured
> twice)?
>
> On 14/06/2019 12:09, Andrea Spina wrote:
>
> Sorry, I totally missed the version: flink-1.6.4, Streaming API
>
> Il giorno ven 14 giu 2019 alle ore 11:08 Chesnay Schepler <
> chesnay@apache.org> ha scritto:
>
>> Which version of Flink are you using? There were some issues at some
>> point about double-counting.
>>
>> On 14/06/2019 09:49, Andrea Spina wrote:
>>
>> Dear Community, I'd like to ask for some details about bytes related
>> metrics in Flink. Precisely, I'm looking at *bytes sent* and *bytes
>> received *metrics: what I am recording is the following:
>>
>> I am reading from a Kafka topic *A* records with schema *K* using a
>> source *Sin *belonging to a pipeline *P1*. The topic *A* is filled with
>> *K* data using a separated pipeline *p2* and a sink *Sout*.
>>
>> Basically what I register is that the *bytes sent* metric of *Sin* - available
>> through the Flink UI -measure twice or more respect of the *bytes
>> received* of *Sout. *For instance, if my sink Sout records 10GB of *bytes
>> received* inbound, then my source Sin emits between 20-25GB *bytes sent*.
>>
>> [image: Screenshot 2019-06-14 at 09.40.08.png]
>>
>> Is someone able to detail how these two metrics are calculated?
>>
>> Thank you,
>>
>> --
>> *Andrea Spina*
>> Software Engineer @ Radicalbit Srl
>> Via Borsieri 41, 20159, Milano - IT
>>
>>
>>
>
> --
> *Andrea Spina*
> Head of R&D @ Radicalbit Srl
> Via Giovanni Battista Pirelli 11, 20124, Milano - IT
>
>
>

-- 
*Andrea Spina*
Head of R&D @ Radicalbit Srl
Via Giovanni Battista Pirelli 11, 20124, Milano - IT

Mime
View raw message