flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fabian Hueske (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-3479) Add hash-based strategy for CombineFunction
Date Tue, 23 Feb 2016 09:35:18 GMT
Fabian Hueske created FLINK-3479:

             Summary: Add hash-based strategy for CombineFunction
                 Key: FLINK-3479
                 URL: https://issues.apache.org/jira/browse/FLINK-3479
             Project: Flink
          Issue Type: Sub-task
          Components: Local Runtime
            Reporter: Fabian Hueske

This issue is similar to FLINK-3477 but adds a hash-based strategy for {{CombineFunction}}
instead of {{ReduceFunction}}.

The interface of {{CombineFunction}} differs from {{ReduceFunction}} by providing an {{Iterable<T>}}
instead of two {{T}} values. Hence, if the {{Iterable<T>}} provides two values, we can
do the same as with a {{ReduceFunction}}.

At the moment, {{CombineFunction}} is wrapped in a {{GroupCombineFunction}} and hence executed
using the {{GroupReduceCombineDriver}}. 
We should add dedicated two dedicated drivers: {{CombineDriver}} and {{ChainedCombineDriver}}
and two driver strategies: {{HASH_COMBINE}} and {{SORT_COMBINE}}. 

If FLINK-3477 is resolved, we can reuse the hash-table.

This message was sent by Atlassian JIRA

View raw message