flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ovidiu-Cristian MARCU <ovidiu-cristian.ma...@inria.fr>
Subject Optimizations not performed - please confirm
Date Tue, 28 Jun 2016 19:41:44 GMT
Hi,

The optimizer internals described in this document [1] are probably not up-to-date.
Can you please confirm if this is still valid:

“The following optimizations are not performed
Join reordering (or operator reordering in general): Joins / Filters / Reducers are not re-ordered
in Flink. This is a high opportunity optimization, but with high risk in the absence of good
estimates about the data characteristics. Flink is not doing these optimizations at this point.
Index vs. Table Scan selection: In Flink, all data sources are always scanned. The data source
(the input format) may apply clever mechanism to not scan all the data, but pre-select and
project. Examples are the RCFile / ORCFile / Parquet input formats."
Any update of this page will be very helpful.

Thank you.

Best,
Ovidiu
[1] https://cwiki.apache.org/confluence/display/FLINK/Optimizer+Internals <https://cwiki.apache.org/confluence/display/FLINK/Optimizer+Internals>
Mime
View raw message