flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-1622) Add GroupReducePartial Operator
Date Tue, 10 Mar 2015 15:15:39 GMT

    [ https://issues.apache.org/jira/browse/FLINK-1622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14355035#comment-14355035
] 

ASF GitHub Bot commented on FLINK-1622:
---------------------------------------

Github user mxm commented on a diff in the pull request:

    https://github.com/apache/flink/pull/466#discussion_r26130398
  
    --- Diff: flink-java/src/main/java/org/apache/flink/api/java/operators/UnsortedGrouping.java
---
    @@ -159,7 +159,23 @@ public UnsortedGrouping(DataSet<T> set, Keys<T> keys)
{
     
     		return new GroupReduceOperator<T, R>(this, resultType, dataSet.clean(reducer),
Utils.getCallLocationName());
     	}
    -	
    +
    +	/**
    +	 * Applies a partial GroupReduce transformation on a grouped {@link DataSet}.
    +	 * In contrast to the reduceGroup transformation, the GroupReduce function is only called
on each partition. Thus,
    +	 * partial solutions are likely to occur.
    +	 * @param reducer The ReduceFunction that is applied on the DataSet.
    +	 * @return A GroupReducePartial operator which represents the partially reduced DataSet
    +	 */
    +	public <R> GroupReducePartialOperator<T, R> reduceGroupPartially(GroupReduceFunction<T,
R> reducer) {
    +		if (reducer == null) {
    +			throw new NullPointerException("GroupReduce function must not be null.");
    +		}
    +		TypeInformation<R> resultType = TypeExtractor.getGroupReduceReturnTypes(reducer,
this.getDataSet().getType());
    +
    +		return new GroupReducePartialOperator<T, R>(this, resultType, dataSet.clean(reducer),
Utils.getCallLocationName());
    +	}
    +
    --- End diff --
    
    See my answer above.


> Add GroupReducePartial Operator
> -------------------------------
>
>                 Key: FLINK-1622
>                 URL: https://issues.apache.org/jira/browse/FLINK-1622
>             Project: Flink
>          Issue Type: Sub-task
>    Affects Versions: 0.9
>            Reporter: Aljoscha Krettek
>
> This does what a Reduce or GroupReduce Operator does, except it is only performed on
a local partition.
> This is also similar to an explicit combine that can output a type that is different
from the input.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message