flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timo Walther <twal...@apache.org>
Subject Re: Problem with SQL-API and nested objects in case class
Date Tue, 21 Nov 2017 09:21:44 GMT
Actually, your use case should be doable with Flink's Table & SQL API 
with some additional UDFs. The API can handle JSON objects if they are 
valid composite types and you can access arrays as well. The splitting 
might be a bit tricky in SQL, you could model it simply as a where() 
clause or maybe a groupBy().

If you experience additional limitations or problems, let us know about it.

Good luck.


Am 11/20/17 um 9:16 PM schrieb Lothium:
> Hey Timo,
> thanks for your warm welcome and for creating a ticket to fix this!
> My scenario is the following:
> I receive different JSON entities from an AMQP queue. I have a source to
> collect the events, after that I parse them into the different internal case
> classes and split the stream via the split function by the entity type.
> After that I want to transform the different entities and write the new JSON
> format (per entity type) to a file sink.
> I know I can do the transformation of the case classes to other case classes
> via scala code, but I thought it would be sometimes easier to do the
> transformations and also aggregations via the SQL or Table-API, but there
> are sometimes JSON arrays with objects in there and this seems to be the
> problem currently.
> I will have a look at the UDFs, maybe I can write something for my purpose.
> Thanks!
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

View raw message