flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jakob Homan <jgho...@gmail.com>
Subject Flink equivalent to Samza's bootstrap stream?
Date Thu, 22 Jun 2017 00:24:12 GMT
Hey all-
   I'm using the Managed Key State to store data in a map.  I would
like, on initial job startup (trigged by a config), for that state to
be populated before processing begings.  This can either be from
another stream or from a file.  In Samza, one would do this with
bootstrap streams
(https://samza.apache.org/learn/documentation/0.13/container/streams.html),
which are consumed entirely before the job begins its normal
processing.

   I'm not seeing an obvious way to accomplish the same thing within
Flink.  The RichFlatMapFunction that is using the MapState could read
a file during the call to open, but it wouldn't appear to know what
subset of keys each instance should consume.

   Any hints?

Thanks,
Jakob

Mime
View raw message