flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From StephanEwen <...@git.apache.org>
Subject [GitHub] incubator-flink pull request: Change Partition Operator to actual ...
Date Fri, 26 Sep 2014 12:54:15 GMT
Github user StephanEwen commented on the pull request:

    I think this variant is a lot simpler and keeps the API clean. Calling partitioning is
something that a user should use, when he wants explicit control, and I think it is okay if
that leads to situations where the partitioning turns out redundant (we could also fix this
later with optimizer rules, but we cannot fix the API once public).
    When calling explicit partitioning before a join, the optimizer will reuse the partitioning.
When calling it before reduce, only the combiner will be dropped.
    Regarding parallelism: Calling rebalancing when the DOP changes is redundant, because
the DOP change will cause a rebalance by itself, but it does not hurt. Calling partition()
and then change the DOP voids the partitioning, that is true. I would accept that for now
and rather throw an error now (e.g. in the optimizer "partition operator followed by DOP change",
and later fix it with an optimizer rule or so) 

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.

View raw message