flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gaurav Luthra <gauravluthra6...@gmail.com>
Subject Re: Partitioning by composite key, But type and number of keys are dynamic
Date Thu, 15 Nov 2018 10:02:17 GMT
Hi Chesnay,

My End user will be aware about the fields of "input records"
(GenericRecord). In configuration my end user only will tell me the name and
number of the fields, based on these fields of GenericRecord I will have to
partition the DataStream and make Keyed Stream.

Currently, I have implemented my solution using KeySelector() function. And
I have converted all the fields mentioned by user into strings, concatenated
these strings (for all the fields for which user want keyBy() means user
wants partitioning) and returned single string from KeySelector() function. 
And partitioning will be happening based on this concatenated string.
See the example below.

                dataStream.keyBy(record ->
                {
                    return
record.get("area").toString()+record.get("age").toString();
                });


But, I am looking for better solution. As I do not want to convert every
field to string.




--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Mime
View raw message