flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ufuk Celebi <...@apache.org>
Subject Re: Flink streaming. Broadcast reference data map across nodes
Date Tue, 21 Feb 2017 14:49:19 GMT
On Tue, Feb 21, 2017 at 2:35 PM, Vadim Vararu <vadim.vararu@adswizz.com> wrote:
> Basically, i have a big dictionary of reference data that has to be
> accessible from all the nodes (in order to do some joins of log line with
> reference line).

If the dictionary is small you can make it part of the closures that
are send to the task managers. Just make it part of your function.

If it is large, I'm not sure what the best way is to do it is right
now. I've CC'd Aljoscha who can probably help...

Mime
View raw message