flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2557) Manual type information via "returns" fails in DataSet API
Date Sun, 23 Aug 2015 19:52:45 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708505#comment-14708505
] 

ASF GitHub Bot commented on FLINK-2557:
---------------------------------------

GitHub user zentol opened a pull request:

    https://github.com/apache/flink/pull/1045

    [FLINK-2557] TypeExtractor properly returns MissingTypeInfo

    This fix is not really obvious so let me explain:
    
    getParameterTye() is called from two different places in the TypeExtractor; to validate
the input type and to extract the output type.
    
    Both cases consider the possibility that getParameterType() fails, but check for different
exceptions. 
    
    The TypeExtractor only returns a MissingTypeInfo if it encounters an InvalidTypesException;
IllegalArgumentExceptions are not catched. This is what @mjsax encountered.
    Changing the exception type causes the TypeExtractor to properly return a MissingTypeInfo,
which is later overridden by the returns(...) call.
    
    In order for the input validation to still work properly aswell, it now catches InvalidTypesExceptions
instead.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/zentol/flink 2557_types

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/1045.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1045
    
----
commit 1c1dc459915c875ab0a4412aa3ef0a844f092171
Author: zentol <s.motsu@web.de>
Date:   2015-08-23T19:41:44Z

    [FLINK-2557] TypeExtractor properly returns MissingTypeInfo

----


> Manual type information via "returns" fails in DataSet API
> ----------------------------------------------------------
>
>                 Key: FLINK-2557
>                 URL: https://issues.apache.org/jira/browse/FLINK-2557
>             Project: Flink
>          Issue Type: Bug
>          Components: Java API
>            Reporter: Matthias J. Sax
>            Assignee: Chesnay Schepler
>
> I changed the WordCount example as below and get an exception:
> Tokenizer is change to this (removed generics and added cast to String):
> {code:java}
> public static final class Tokenizer implements FlatMapFunction {
> 	public void flatMap(Object value, Collector out) {
> 		String[] tokens = ((String) value).toLowerCase().split("\\W+");
> 		for (String token : tokens) {
> 			if (token.length() > 0) {
> 				out.collect(new Tuple2<String, Integer>(token, 1));
> 			}
>  		}
> 	}
> }
> {code}
> I added call to "returns(....)" here:
> {code:java}
> DataSet<Tuple2<String, Integer>> counts =
> 	text.flatMap(new Tokenizer()).returns("Tuple2<String,Integer>")
> 	.groupBy(0).sum(1);
> {code}
> The exception is:
> {noformat}
> Exception in thread "main" java.lang.IllegalArgumentException: The types of the interface
org.apache.flink.api.common.functions.FlatMapFunction could not be inferred. Support for synthetic
interfaces, lambdas, and generic types is limited at this point.
>  	at org.apache.flink.api.java.typeutils.TypeExtractor.getParameterType(TypeExtractor.java:686)
>  	at org.apache.flink.api.java.typeutils.TypeExtractor.getParameterTypeFromGenericType(TypeExtractor.java:710)
>  	at org.apache.flink.api.java.typeutils.TypeExtractor.getParameterType(TypeExtractor.java:673)
>  	at org.apache.flink.api.java.typeutils.TypeExtractor.privateCreateTypeInfo(TypeExtractor.java:365)
>  	at org.apache.flink.api.java.typeutils.TypeExtractor.getUnaryOperatorReturnType(TypeExtractor.java:279)
>  	at org.apache.flink.api.java.typeutils.TypeExtractor.getFlatMapReturnTypes(TypeExtractor.java:120)
>  	at org.apache.flink.api.java.DataSet.flatMap(DataSet.java:262)
>  	at org.apache.flink.examples.java.wordcount.WordCount.main(WordCount.java:69)
> {noformat}
> Fix:
> This should not immediately fail, but also only give a "MissingTypeInfo" so that type
hints would work.
> The error message is also wrong, btw: It should state that raw types are not supported.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message