flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: Random access to small global state
Date Thu, 14 Jul 2016 14:31:54 GMT
Hi,
For Ignite, Flink has a Sink, which is a one-directional thing.

I think that Sebastian needs a bi-directional connection.
An in-memory KV store like redis or memcache is probably the best option
for such a use case (it reminds me a bit of the Yahoo streaming benchmark
[1]).

[1]
https://yahooeng.tumblr.com/post/135321837876/benchmarking-streaming-computation-engines-at


On Sat, Jul 9, 2016 at 4:03 PM, Suneel Marthi <suneel.marthi@gmail.com>
wrote:

> U could use ignite too, I believe they have a plugin for flink streaming.
>
> Sent from my iPhone
>
> > On Jul 9, 2016, at 8:05 AM, Sebastian <ssc@apache.org> wrote:
> >
> > Hi,
> >
> > I'm planning to work on a streaming recommender in Flink, and one
> problem that I have is that the algorithm needs random access to a small
> global state (say a million counts). It should be ok if there is some
> inconsistency in the state (e.g., delay in seeing updates).
> >
> > Does anyone here have experience with such things? I'm thinking of
> connecting Flink to a lighweight in-memory key-value store such as memcache
> for that.
> >
> > Best,
> > Sebastian
>

Mime
View raw message