flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Optimizations not performed - please confirm
Date Wed, 29 Jun 2016 15:27:37 GMT
I think this document is still up-to-date since not much was done in these
parts of the code for the 1.0 release and after that.

Maybe Timo can give some insights into what optimizations are done in the
Table API/SQL that will be be released in an updated version in 1.1.


+Timo, Explicitly adding Timo

On Tue, 28 Jun 2016 at 21:41 Ovidiu-Cristian MARCU <
ovidiu-cristian.marcu@inria.fr> wrote:

> Hi,
> The optimizer internals described in this document [1] are probably not
> up-to-date.
> Can you please confirm if this is still valid:
> *“The following optimizations are not performed*
>    - *Join reordering (or operator reordering in general): Joins /
>    Filters / Reducers are not re-ordered in Flink. This is a high opportunity
>    optimization, but with high risk in the absence of good estimates about the
>    data characteristics. Flink is not doing these optimizations at this point.*
>    - *Index vs. Table Scan selection: In Flink, all data sources are
>    always scanned. The data source (the input format) may apply clever
>    mechanism to not scan all the data, but pre-select and project. Examples
>    are the RCFile / ORCFile / Parquet input formats."*
> Any update of this page will be very helpful.
> Thank you.
> Best,
> Ovidiu
> [1] https://cwiki.apache.org/confluence/display/FLINK/Optimizer+Internals

View raw message