flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tobias (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-925) Support KeySelector function returning Tuples
Date Fri, 27 Jun 2014 13:24:24 GMT

    [ https://issues.apache.org/jira/browse/FLINK-925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14045928#comment-14045928

Tobias commented on FLINK-925:

When I have a reduce function: ReduceFunction<Tuple2<Tuple2<String, Integer>,
I get an exception:
{color: red}
Exception in thread "main" java.lang.IllegalArgumentException: The field at position 0 (Tuple2<String,
Integer>) is no atomic key type.
at eu.stratosphere.api.java.typeutils.TupleTypeInfo.createLeadingFieldComparator(TupleTypeInfo.java:225)
	at eu.stratosphere.api.java.typeutils.TupleTypeInfo.createComparator(TupleTypeInfo.java:117)
	at eu.stratosphere.compiler.postpass.JavaApiPostPass.createComparator(JavaApiPostPass.java:322)

Creating a comparator for the Tuple type fails. 

Might coGroup not be restricted because it does not work with KeySelectors?

I will look into coGroup and see if and how it works there.

> Support KeySelector function returning Tuples
> ---------------------------------------------
>                 Key: FLINK-925
>                 URL: https://issues.apache.org/jira/browse/FLINK-925
>             Project: Flink
>          Issue Type: Improvement
>    Affects Versions: 0.6-incubating
>            Reporter: Fabian Hueske
>            Assignee: Tobias
>            Priority: Minor
>              Labels: starter
> KeySelector functions are used to extract keys on which DataSets can be grouped or joined.
> Currently, the keys types returned by KeySelector function are restricted to be comparable.
However, Flinks Tuple data types are not comparable (because this depends on the types of
its fields) which makes grouping and joining on composite keys difficult.
> We should change the signature of the groupBy(), join(), and coGroup() methods to allow
also non-comparable keys as return types of a KeySelector function. 
> Instead we will check at optimization time whether the returned type is comparable (which
is true for tuples if all elements are comparable).

This message was sent by Atlassian JIRA

View raw message