flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Felix Neutatz <neut...@googlemail.com>
Subject [DISCUSS] FLIP-5 Only send data to each taskmanager once for broadcasts
Date Fri, 22 Jul 2016 12:59:59 GMT
Hi everybody,

I want to improve the performance of broadcasts in Flink. Therefore Till
told me to start a FLIP on this topic to discuss how to go forward to solve
the current issues for broadcasts.

The problem in a nutshell: Instead of sending data to each taskmanager only
once, at the moment the data is sent to each task. This means if there are
3 slots on each taskmanager we will send the data 3 times instead of once.

There are multiple ways to tackle this problem and I started to do some
research and investigate. You can follow my thought process here:

https://cwiki.apache.org/confluence/display/FLINK/FLIP-5%3A+Only+send+data+to+each+taskmanager+once+for+broadcasts

This is my first FLIP. So please correct me, if I did something wrong.

I am interested in your thoughts about how to solve this issue. Do you
think my approach is heading into the right direction or should we follow a
totally different one.

I am happy about any comment :)

Best regards,
Felix

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message