flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Simone Robutti <simone.robu...@radicalbit.io>
Subject Re: Java Maps and Type Information
Date Wed, 02 Mar 2016 10:12:57 GMT
Ok, I made it work but there's still an issue. I used
.returns(java.util.Map.class) after the "map" call and it works with this
simple function but it doesn't compile with my CustomMapFunction that
extends MapFunction. It gives a compilation error on the .returns() call.

This is the case only if the variable operator is of type CustomMapFunction
but if I do

> MapFunction operator = new CustomMapFunction();

it works again.

If I go back to

> CustomMapFunction operator = new CustomMapFunction();

it gives this error:

>Error:(43, 87) java: no suitable method found for
returns(java.lang.Class<java.util.Map>)
    method

Should I open an issue?

2016-03-01 21:45 GMT+01:00 Simone Robutti <simone.robutti@radicalbit.io>:

> I tried to simplify it to the bones but I'm actually defining a custom
> MapFunction<java.util.Map<String,Object>,java.util.Map<String,Object>>
that
> even with a simple identity function fails at runtime giving me the
> following error:
>
> >Exception in thread "main"
> org.apache.flink.api.common.functions.InvalidTypesException: The return
> type of function 'main(Main.java:45)' could not be determined
> automatically, due to type erasure. You can give type information hints by
> using the returns(...) method on the result of the transformation call, or
> by letting your function implement the 'ResultTypeQueryable' interface.
>
> where the line 45 is the line where I invoke the map function.
>
> Here the piece of code:
>
> ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
>         env.setParallelism(2);
>         Map<String,Object> inputMap = new HashMap<String,Object>();
>         inputMap.put("sepal_width",2.0);
>         inputMap.put("sepal_length",2.0);
>         inputMap.put("petal_width",2.0);
>         inputMap.put("petal_length",2.0);
>
>         MapFunction operator=new
> MapFunction<Map<String,Object>,Map<String,Object>>(){
>
>             @Override
>             public Map<String, Object> map(Map<String, Object>
> stringObjectMap) throws Exception {
>                 return stringObjectMap;
>             }
>         };
>
>         List<Map<String, Object>> input = new LinkedList<>();
>         input.add(inputMap);
>         DataSource<Map<String, Object>> dataset =
> env.fromCollection(input);
>         List<java.util.Map<FieldName, Object>> collectedResult =
> dataset.map(operator).collect();
>
>
>
>
> 2016-03-01 16:42 GMT+01:00 Aljoscha Krettek <aljoscha@apache.org>:
>
>> Hi,
>> what kind of program are you writing? I just wrote a quick example using
>> the DataStream API where I’m using Map<String, Tuple2<String, Integer>>
as
>> the output type of one of my MapFunctions.
>>
>> Cheers,
>> Aljoscha
>> > On 01 Mar 2016, at 16:33, Simone Robutti <simone.robutti@radicalbit.io>
>> wrote:
>> >
>> > Hello,
>> >
>> > to my knowledge is not possible to use a java.util.Map for example in a
>> FlatMapFunction<java.util.Map, java.util.Map>. Is that correct? It gives a
>> typer error at runtime and it doesn't work even with explicit
>> TypeInformation hints.
>> >
>> > Is there any way to make it work?
>> >
>> > Thanks,
>> >
>> > Simone
>>
>>
>

Mime
View raw message