flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2897) Use distinct initial indices for OutputEmitter round-robin
Date Fri, 23 Oct 2015 13:12:27 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14970936#comment-14970936
] 

ASF GitHub Bot commented on FLINK-2897:
---------------------------------------

Github user greghogan commented on a diff in the pull request:

    https://github.com/apache/flink/pull/1292#discussion_r42862734
  
    --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/operators/shipping/OutputEmitter.java
---
    @@ -143,14 +148,25 @@ public OutputEmitter(ShipStrategyType strategy, TypeComparator<T>
comparator, Pa
     	
     	// --------------------------------------------------------------------------------------------
     
    +	private int[] forward() {
    +		if (this.channels == null) {
    --- End diff --
    
    Yes, I did overlook that, though customPartition does likewise and the OutputEmitter will
only be used for a single strategy. The strategy is set in the constructor so the channel
list can be created there for the types which do not broadcast.


> Use distinct initial indices for OutputEmitter round-robin
> ----------------------------------------------------------
>
>                 Key: FLINK-2897
>                 URL: https://issues.apache.org/jira/browse/FLINK-2897
>             Project: Flink
>          Issue Type: Improvement
>          Components: Distributed Runtime
>    Affects Versions: 0.10
>            Reporter: Greg Hogan
>            Assignee: Greg Hogan
>
> Currently, when performing a round-robin partitioning each task will sequentially partition
starting with partition "1". This is fine in the usual case where the number of partitioned
objects greatly exceeds the number of channels. However, in the case where the number of objects
is relatively few (each, perhaps, requiring a large computation or access to an external system)
it would be much better to begin partitioning at distinct indices (the task index).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message