flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dulce Morim <Dulce.Mo...@i2s.pt>
Subject Keyby connect for a one to many relationship - DataStream API - Ride Enrichment (CoProcessFunction)
Date Mon, 26 Mar 2018 15:18:18 GMT
Hello,

Following this exercise:
http://training.data-artisans.com/exercises/rideEnrichment-processfunction.html

I need to do something similar, but my data structure is something like:

A
Primary_key
other fields

B
Primary_key
Relation_Key
other fields

Where A and B relationship is one to more, on B.Relation_key = A.Primary_key

When using the keyby function on both streams, with the key "A.Primary_key" on the A stream
and the "B.Relation_key" on the B stream, the data that comes from B, only shows the last
occurrence of the records that had the same "B.Relation_key".

Is it possible to connect these two streams? In this solution there seems to be a 1 to 1 relationship,
but we want a one to many relationship. Should this be solved via another process?

Thanks,
Dulce Morim
Mime
View raw message