flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Márton Balassi <balassi.mar...@gmail.com>
Subject Re: throttled stream
Date Sun, 17 Apr 2016 06:53:33 GMT
There is a utility in flink-streaming-examples that might be useful, but is
generally the same idea that Niels suggests. [1]

[1]
https://github.com/apache/flink/blob/master/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/utils/ThrottledIterator.java

On Sun, Apr 17, 2016 at 8:42 AM, Niels Basjes <niels@basj.es> wrote:

> Simple idea: create a map function that only does "sleep 1/5 second" and
> put that in your pipeline somewhere.
>
> Niels
> On 16 Apr 2016 22:38, "Chen Bekor" <chen.bekor@gmail.com> wrote:
>
>> is there a way to consume a kafka stream using flink with  a predefined
>> rate limit (eg 5 events per second)
>>
>> we need this because we need to control some 3rd party api rate
>> limitations so,  even if we have a much larger throughput potential, we
>> must control the consumption rate in order not to overflow the API channel.
>>
>

Mime
View raw message