flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aljoscha <...@git.apache.org>
Subject [GitHub] flink pull request: [FLINK-3336] Add Rescale Data Shipping for Dat...
Date Fri, 05 Feb 2016 10:20:01 GMT
Github user aljoscha commented on the pull request:

    https://github.com/apache/flink/pull/1589#issuecomment-180284761
  
    I addressed @uce's comments, thanks for those. I also had a discussion with @StephanEwen
and he came up with the name `rescale()` for this distribution pattern. The reasoning is that
you basically increase or decrease/increase the parallelism of downstream operations by some
factor x and elements from the upstream operation will be sent only to x downstream operations
to which no other upstream operation sends elements. It should be clear from the picture what
I mean by that exactly. We have a source with parallelism 3, then a map with parallelism 9
and then the sink has parallelism 3 again.
    <img width="1986" alt="screen shot 2016-02-05 at 11 18 16" src="https://cloud.githubusercontent.com/assets/68551/12843516/3d509ab6-cbfa-11e5-850d-d4cd25f08de5.png">
      


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message