flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Knauf <konstantin.kn...@tngtech.com>
Subject Re: Enriching a tuple mapped from a datastream with data coming from a JDBC source
Date Mon, 12 Sep 2016 05:46:13 GMT
Hi Philipp,

the easist way is a RichMap. In the open()-Method you can load the
relevant database table into memory (e.g. a HashMap). In the
map()-method you than just look up the entry in the HashMap.

Of course, this only works if the dataset is small enough to fit in
memory. Is it?



On 12.09.2016 02:36, Philipp Bussche wrote:
> Hi there,
> I have a data stream (coming from Kafka) that contains information which
> I want to enrich with information that sits in a database before I
> handover the enriched tuple to a sink.
> How would I do that ?
> I was thinking of somehow combining my streaming job with a JDBC input
> but wasn't very succesful in getting this going.
> Thanks
> Philipp

Konstantin Knauf * konstantin.knauf@tngtech.com * +49-174-3413182
TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterföhring
Geschäftsführer: Henrik Klagges, Christoph Stock, Dr. Robert Dahlke
Sitz: Unterföhring * Amtsgericht München * HRB 135082

View raw message