flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: CSV input with unknown # of fields and Custom output format
Date Wed, 04 Feb 2015 11:50:01 GMT
Nice!

BTW: The TypeSerializerInputFormat just changed (in the 0.9-SNAPSHOT
master) so that it now takes the type information, rather than a type
serializer...

Stephan


On Wed, Feb 4, 2015 at 11:52 AM, Vinh June <hoangthevinh.htv@gmail.com>
wrote:

> Thanks,
> I just tried and it works with scala also.
>
> Small notice for anyone who mights interested is that the constructor of
> TypeSerializerInputFormat needs a TypeSerializer, not a TypeInformation. So
> this would work in Scala:
> ----------------------------------------
> [SCALA]
> val readback = env
>   .readFile[MyClass](
>     new
> TypeSerializerInputFormat[MyClass](dataSet.getType.createSerializer()),
>     dataPath)
> ----------------------------------------
>
> If you want to separate write and readFile into different object (as in
> above code, I used env.readFile ), then in this case, dataSet is not
> available, we'll need to create serializer from class definition as below
> ----------------------------------------
> [SCALA]
> val readback = env
>   .readFile[MyClass](
>     new
>
> TypeSerializerInputFormat[MyClass](createTypeInformation[MyClass].createSerializer()),
>     dataPath)
> ----------------------------------------
>
> Again, thank you @Stephan
>
>
>
> --
> View this message in context:
> http://apache-flink-incubator-user-mailing-list-archive.2336050.n4.nabble.com/CSV-input-with-unknown-of-fields-and-Custom-output-format-tp670p675.html
> Sent from the Apache Flink (Incubator) User Mailing List archive. mailing
> list archive at Nabble.com.
>

Mime
View raw message