flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Igor Berman <igor.ber...@gmail.com>
Subject Re: Flink event processing immediate feedback
Date Thu, 14 Apr 2016 15:45:58 GMT
Yes, indeed this is direction we are trying currently

thanks


On 14 April 2016 at 18:31, Aljoscha Krettek <aljoscha@apache.org> wrote:

> Could still be, as I described it by using a message queue to do the
> communication between Flink and the front end.
>
> On Thu, 14 Apr 2016 at 17:30 Igor Berman <igor.berman@gmail.com> wrote:
>
>> Hi Aljoscha,
>> thanks for the response
>>
>> Synchronous - in our case means that request by end-client to
>> frontend(say some REST call) needs to wait until processing in
>> backend(Flink) is done and should return response(e.g. alert) back to
>> end-client(i.e. end-client -> frontend -> kafka-> flink)
>> those request are minority and will be defined by different type of
>> request to frontend or different parameters
>>
>> Might be Flink is not best option for this use case?
>>
>> Igor
>>
>>
>>
>>
>>
>> On 14 April 2016 at 16:37, Aljoscha Krettek <aljoscha@apache.org> wrote:
>>
>>> Hi,
>>> what do you mean by "synchronous". If I understood it correctly then
>>> some events entering the Flink pipeline would trigger an alert while some
>>> others would not trigger an alert. How would the component that receives
>>> such alerts know when to wait and when to don't wait.
>>>
>>>  As I see it you can push these alerts into some sort of message queue
>>> and have a request in the front-end that returns as soon as something is
>>> available in that queue. Then you display it and start a new query that
>>> returns as soon as more data is available in the alert queue.
>>>
>>> Cheers,
>>> Aljoscha
>>>
>>> On Thu, 7 Apr 2016 at 09:57 igor.berman <igor.berman@gmail.com> wrote:
>>>
>>>> Hi,
>>>> Suppose I have web facing frontend that gets stream of events(http
>>>> calls). I
>>>> need to process event stream and do some aggregations over those events
>>>> and
>>>> write aggregated statistics to Hbase - so far Flink seems as perfect
>>>> match.
>>>> However in some cases event should trigger some alert and frontend
>>>> needs to
>>>> get this alert in synchronous way - here I'm a bit lost. I thought about
>>>> some kind of following flow:
>>>> frontend -> queue -> flink -> redis(pub/sub)<- frontend
>>>>
>>>> I.e. I have two major use cases - async aggregated analytics/stats
>>>> computing
>>>> and "synchronous" response to frontend. Frontend might be node/play or
>>>> any
>>>> other technology that won't have a problem of "waiting" for the
>>>> response, so
>>>> the only question - how to implement this feedback ?
>>>> Might be some kind of Sink?
>>>>
>>>> Any ideas would be appreciated,
>>>> Igor
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-event-processing-immediate-feedback-tp5978.html
>>>> Sent from the Apache Flink User Mailing List archive. mailing list
>>>> archive at Nabble.com.
>>>>
>>>
>>

Mime
View raw message