flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Alexandrov <alexander.s.alexand...@gmail.com>
Subject Re: how Flink Optimizer work and what is process do it?
Date Mon, 25 May 2015 21:42:15 GMT
The flink optimizer will basically do two things for you (+ some other
magic regarding iterations which I will leave aside for now)

1) Algorithm selection: e.g., choose between a  hash-join or merge-join;
2) Partitioning strategy: track grouping key, and if partitioning is
needed, pick a suitable strategy (e.g. re-partition vs broadcast for a
join);

Besides that the optimizer will fix the hyperparameters for your algorithms
(e.g. memory allocation) and will ultimately generate the final program
that has to be executed.

[1] http://en.wikipedia.org/wiki/Hash_join
[2] http://en.wikipedia.org/wiki/Sort-merge_join

2015-05-25 20:27 GMT+02:00 hagersaleh <loveallah1987@yahoo.com>:

> how Flink Optimizer work and what is process do it?
>
>
>
> --
> View this message in context:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/how-Flink-Optimizer-work-and-what-is-process-do-it-tp1359.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> at Nabble.com.
>

Mime
View raw message