flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Looking up values in a metadata store for "condensed" events
Date Wed, 18 May 2016 10:52:46 GMT
Hi,
what you can also do is use a RichMapFunction. Rich functions have
open()/close() methods that get called before the first element and the
last element, respectively.
See here
https://ci.apache.org/projects/flink/flink-docs-master/apis/common/index.html#specifying-transformation-functions
for
more information.

Cheers,
Aljoscha

On Tue, 10 May 2016 at 07:31 milind parikh <milindsparikh@gmail.com> wrote:

> I am new to flink. I am wondering what the best practice is for looking up
> additional values for entities embedded in an event.
>
> For example, an event might have the engine model #. A metadata store has
> the typical rpm characteristics of an engine model #; which can be used in
> a subsequent processing of the event. I call the process of event
> inflation  "event adornment". Currently I use a Map function transformation
> to do the lookup in the Map function through instatiating a connection in
> the Map function to the Metadata store per map execution. The thing works;
> however having 100K connections is not very ideal (assuming 100k eps). Any
> alternative ideas; given the implementing class has to be Serializable?
>

Mime
View raw message