flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chesnay Schepler <ches...@apache.org>
Subject Re: Flink UI records received/sent chained-unchained operators
Date Wed, 24 May 2017 08:29:25 GMT
Hello,

I will try to reproduce this later today. My current assumption is that 
the aggregation logic in the webUI for some reason adds the wrong counters,
it's unlikely that the counts are matching just by chance.

Regards,
Chesnay

On 24.05.2017 10:15, Luis Lázaro wrote:
> Hi everyone,
> my use case:
>
> • Flink 1.3.0-SNAPSHOT
> • DataStream API.
> • Cluster mode is standalone, 3 TM and a JM.
> • Parallelism is set to 1 on client level from UI.
> • Source is Kafka configured with one single partition.
>
>  In Flink’s UI counter, I am getting a record count “sent" greater 
> than the count of entries “received".Despite this, _total records is 
> correct, ie, neither I losing nor am I getting more records._ Please 
> check out screenshot#1:
>
>
>
>  My question is: *why the total number of records sent minus received 
> is exactly the total leaked?* (250-223 = 27), i expected to see 27 
> "Records sent”
> ¿ Is Map->Filter-Filter, as a single operator a /SplitStream /?
>
> /.split(
> case false => 223
> case true => 27
> ) /
>
> If I run _another job_ with the same execution plan but unchained 
> operators in global operator (MAP->FILTER->FILTER) , the account is 
> exactly the same records sent to the account filtered records. Please 
> check out screenshot#2
>
>
>
>
> PD: There is a similar question in Flink UI and records sent 
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-UI-and-records-sent-td12522.html>
, 
> but I think they are not related.
>
> Thanks in advance, any suggestion is well appreciated.
>
> Best regards, Luis
>
>
>


Mime
View raw message