flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: Dealing with latency in Sink
Date Tue, 07 Feb 2017 16:18:45 GMT
Hi Mohit,

Flink doesn't allow dynamic up or downscaling of parallel operator
instances at runtime. However, you can stop and restore from a savepoint
with a different parallelism. This way, you can adopt to workload changes.

Flink's handling of backpressure is very implicit. If you want to throttle
a sink, you can just accept fewer records per second (by not returning from
the invoke() call) to throttle the entire topology.

I think we can provide you much better support if you can provide us with
some more details on your problem (which system do you want to connect to,
what are your latency / throughput requirements, how much does the load
change)

On Mon, Feb 6, 2017 at 10:34 PM, Mohit Anchlia <mohitanchlia@gmail.com>
wrote:

> What is the best way to dynamically adapt and tune down number of tasks
> created to write/read to a sink when sink slows down or the latency to sink
> increases? I am looking at the sink interface but don't see a way to
> influence flink to reduce the number of tasks or throttle the volume down
> to the sink. What is the best way to deal with this scenario?
>

Mime
View raw message