flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Alexandrov <alexander.s.alexand...@gmail.com>
Subject Re: Fwd: TypeSerializerInputFormat cannot determine its type automatically
Date Thu, 29 Jan 2015 15:21:54 GMT
Alight, thanks for the hint.

I suggest to close PR 349 and refine the exception with a hint HOW exactly
to pass the TypeInformation instance, e.g.

The type returned by the input format could not be automatically
determined. Please pass the TypeInformation of the produced type explicitly
via 'env.createInput(...)'.

I knew what I had to do, but I couldn't find the right point of entry to do
is because the IO system is so generic.


2015-01-29 16:07 GMT+01:00 Timo Walther <twalthr@apache.org>:

> Hey Alexander,
>
> I have looked into your issue. You can simply use
> env.createInput(InputFormat,TypeInformation) instead of env.readFile()
> then you can pass TypeInformation manually without implementing
> ResultTypeQueryable.
>
> Regards,
> Timo
>
>
>
>
> On 29.01.2015 14:54, Alexander Alexandrov wrote:
>
>> The problem seems to be that the reflection analysis cannot determine the
>> type of the TypeSerializerInputFormat.
>>
>> One possible solution is to add the ResultTypeQueryable interface and
>> force
>> clients to explicitly set the TypeInformation.
>>
>> This might break code which relies on automatic type inference, but at the
>> moment I cannot find any other usages of the TypeSerializerInputFormat
>> except from the unit test.
>>
>>
>> ---------- Forwarded message ----------
>> From: Alexander Alexandrov <alexander.s.alexandrov@gmail.com>
>> Date: 2015-01-29 12:04 GMT+01:00
>> Subject: TypeSerializerInputFormat cannot determine its type automatically
>> To: user@flink.apache.org
>>
>>
>> I am trying to use the TypeSerializer IO formats to write temp data to
>> disk. A gist with a minimal example can be found here:
>>
>> https://gist.github.com/aalexandrov/90bf21f66bf604676f37
>>
>> However, with the current setting I get the following error with the
>> TypeSerializerInputFormat:
>>
>> Exception in thread "main"
>> org.apache.flink.api.common.InvalidProgramException: The type returned by
>> the input format could not be automatically determined. Please specify the
>> TypeInformation of the produced type explicitly.
>>      at
>> org.apache.flink.api.java.ExecutionEnvironment.readFile(
>> ExecutionEnvironment.java:341)
>>      at SerializedFormatExample$.main(SerializedFormatExample.scala:48)
>>      at SerializedFormatExample.main(SerializedFormatExample.scala)
>>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>      at
>> sun.reflect.NativeMethodAccessorImpl.invoke(
>> NativeMethodAccessorImpl.java:57)
>>      at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(
>> DelegatingMethodAccessorImpl.java:43)
>>      at java.lang.reflect.Method.invoke(Method.java:606)
>>      at com.intellij.rt.execution.application.AppMain.main(
>> AppMain.java:134)
>>
>> I think that the typeInformation instance at line 43 should be somehow
>> passed to the TypeSerializerInputFormat, but I cannot find a way to do it.
>>
>> Any suggestions?
>>
>> Thanks,
>> A.
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message