flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2557) Manual type information via "returns" fails in DataSet API
Date Sun, 23 Aug 2015 19:52:45 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708505#comment-14708505

ASF GitHub Bot commented on FLINK-2557:

GitHub user zentol opened a pull request:


    [FLINK-2557] TypeExtractor properly returns MissingTypeInfo

    This fix is not really obvious so let me explain:
    getParameterTye() is called from two different places in the TypeExtractor; to validate
the input type and to extract the output type.
    Both cases consider the possibility that getParameterType() fails, but check for different
    The TypeExtractor only returns a MissingTypeInfo if it encounters an InvalidTypesException;
IllegalArgumentExceptions are not catched. This is what @mjsax encountered.
    Changing the exception type causes the TypeExtractor to properly return a MissingTypeInfo,
which is later overridden by the returns(...) call.
    In order for the input validation to still work properly aswell, it now catches InvalidTypesExceptions

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/zentol/flink 2557_types

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1045
commit 1c1dc459915c875ab0a4412aa3ef0a844f092171
Author: zentol <s.motsu@web.de>
Date:   2015-08-23T19:41:44Z

    [FLINK-2557] TypeExtractor properly returns MissingTypeInfo


> Manual type information via "returns" fails in DataSet API
> ----------------------------------------------------------
>                 Key: FLINK-2557
>                 URL: https://issues.apache.org/jira/browse/FLINK-2557
>             Project: Flink
>          Issue Type: Bug
>          Components: Java API
>            Reporter: Matthias J. Sax
>            Assignee: Chesnay Schepler
> I changed the WordCount example as below and get an exception:
> Tokenizer is change to this (removed generics and added cast to String):
> {code:java}
> public static final class Tokenizer implements FlatMapFunction {
> 	public void flatMap(Object value, Collector out) {
> 		String[] tokens = ((String) value).toLowerCase().split("\\W+");
> 		for (String token : tokens) {
> 			if (token.length() > 0) {
> 				out.collect(new Tuple2<String, Integer>(token, 1));
> 			}
>  		}
> 	}
> }
> {code}
> I added call to "returns(....)" here:
> {code:java}
> DataSet<Tuple2<String, Integer>> counts =
> 	text.flatMap(new Tokenizer()).returns("Tuple2<String,Integer>")
> 	.groupBy(0).sum(1);
> {code}
> The exception is:
> {noformat}
> Exception in thread "main" java.lang.IllegalArgumentException: The types of the interface
org.apache.flink.api.common.functions.FlatMapFunction could not be inferred. Support for synthetic
interfaces, lambdas, and generic types is limited at this point.
>  	at org.apache.flink.api.java.typeutils.TypeExtractor.getParameterType(TypeExtractor.java:686)
>  	at org.apache.flink.api.java.typeutils.TypeExtractor.getParameterTypeFromGenericType(TypeExtractor.java:710)
>  	at org.apache.flink.api.java.typeutils.TypeExtractor.getParameterType(TypeExtractor.java:673)
>  	at org.apache.flink.api.java.typeutils.TypeExtractor.privateCreateTypeInfo(TypeExtractor.java:365)
>  	at org.apache.flink.api.java.typeutils.TypeExtractor.getUnaryOperatorReturnType(TypeExtractor.java:279)
>  	at org.apache.flink.api.java.typeutils.TypeExtractor.getFlatMapReturnTypes(TypeExtractor.java:120)
>  	at org.apache.flink.api.java.DataSet.flatMap(DataSet.java:262)
>  	at org.apache.flink.examples.java.wordcount.WordCount.main(WordCount.java:69)
> {noformat}
> Fix:
> This should not immediately fail, but also only give a "MissingTypeInfo" so that type
hints would work.
> The error message is also wrong, btw: It should state that raw types are not supported.

This message was sent by Atlassian JIRA

View raw message