flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: Usecase for Flink
Date Fri, 18 Dec 2015 12:47:39 GMT
If I understand you correctly, you want to write something like:

------------------------------------------------------------------

                                                    [cassandra]
                              ^
                              |
                              V
(event source) ----> (Add event and lookup) ---> (further ops)

------------------------------------------------------------------

That should work with Flink, yes. You can communicate with an external
Cassandra service inside functions.

We are also working on making larger-than-memory state easily supported in
Flink, so future versions may allow you to
do this without any external service.






On Thu, Dec 17, 2015 at 8:54 AM, igor.berman <igor.berman@gmail.com> wrote:

> Hi,
> We are looking at Flink and trying to understand if our usecase is relevant
> to it.
>
> We need process stream of events. Each event is for some id(e.g. device
> id),
> when each event should be
> 1. stored in some persistent storage(e.g. cassandra)
> 2. previously persisted events should be fetched and some computation over
> whole history may or may not trigger some other events(e.g. sending email)
>
> so yes we have stream of events, but we need persistent store(aka external
> service) in the middle
> and there is no aggregation of those events into something smaller which
> could be stored in memory, i.e. number of ids might be huge and previous
> history of events per each id can be considerable so that no way to store
> everything in memory
>
> I was wondering if akka stream is sort of optional solution too
>
> please share your ideas :)
> thanks in advance,
> Igor
>
>
>
> --
> View this message in context:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Usecase-for-Flink-tp4076.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> at Nabble.com.
>

Mime
View raw message