flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Igor Berman <igor.ber...@gmail.com>
Subject Re: Flink event processing immediate feedback
Date Thu, 14 Apr 2016 15:29:54 GMT
Hi Aljoscha,
thanks for the response

Synchronous - in our case means that request by end-client to frontend(say
some REST call) needs to wait until processing in backend(Flink) is done
and should return response(e.g. alert) back to end-client(i.e. end-client
-> frontend -> kafka-> flink)
those request are minority and will be defined by different type of request
to frontend or different parameters

Might be Flink is not best option for this use case?

Igor





On 14 April 2016 at 16:37, Aljoscha Krettek <aljoscha@apache.org> wrote:

> Hi,
> what do you mean by "synchronous". If I understood it correctly then some
> events entering the Flink pipeline would trigger an alert while some others
> would not trigger an alert. How would the component that receives such
> alerts know when to wait and when to don't wait.
>
>  As I see it you can push these alerts into some sort of message queue and
> have a request in the front-end that returns as soon as something is
> available in that queue. Then you display it and start a new query that
> returns as soon as more data is available in the alert queue.
>
> Cheers,
> Aljoscha
>
> On Thu, 7 Apr 2016 at 09:57 igor.berman <igor.berman@gmail.com> wrote:
>
>> Hi,
>> Suppose I have web facing frontend that gets stream of events(http
>> calls). I
>> need to process event stream and do some aggregations over those events
>> and
>> write aggregated statistics to Hbase - so far Flink seems as perfect
>> match.
>> However in some cases event should trigger some alert and frontend needs
>> to
>> get this alert in synchronous way - here I'm a bit lost. I thought about
>> some kind of following flow:
>> frontend -> queue -> flink -> redis(pub/sub)<- frontend
>>
>> I.e. I have two major use cases - async aggregated analytics/stats
>> computing
>> and "synchronous" response to frontend. Frontend might be node/play or any
>> other technology that won't have a problem of "waiting" for the response,
>> so
>> the only question - how to implement this feedback ?
>> Might be some kind of Sink?
>>
>> Any ideas would be appreciated,
>> Igor
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-event-processing-immediate-feedback-tp5978.html
>> Sent from the Apache Flink User Mailing List archive. mailing list
>> archive at Nabble.com.
>>
>

Mime
View raw message