flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Key factors for Flink's performance
Date Wed, 11 May 2016 07:24:17 GMT
Hi,
latency for Flink and Storm are pretty similar. The only reason I could see
for Flink having the slight upper hand there is the fact that Storm tracks
the progress of every tuple throughout the topology and requires ACKs that
have to go back to the sinks.

As for throughput you are right that Flink sends elements in batches. The
size of these batches can be controlled, even be reduced to 1, which yields
best latency. The fact that there are these batches not not visible
anywhere in the model, so calling them micro batches is problematic, since
that already refers to a very different concept in Spark Streaming.

Cheers,
Aljoscha

On Mon, 9 May 2016 at 11:06 <leon_mclare@tutanota.com> wrote:

> Hello Flink team,
>
> i am currently playing around with Storm and Flink in the context of a
> smart home. The primary functional requirement is to quickly react to
> certain properties in stream tuples.
>
> I was looking at some benchmarks from the two systems, and generally Flink
> has the upper hand, in both throughput and latency. I do not really
> understand how Flink achieves better latency than Storm, which is driven by
> one-at-at-time tuples.
>
> From what i understood in the documentation, Flink performs micro batching
> when transferring data across the network to downstream operators located
> on other nodes. Perhaps this achieves a better average latency.
>
> Surely the bigger factor however is that Flink can completely bypass
> internal operator queues with operator chaining, which Storm cannot do.
>
> Kind regards
> Leon
> <https://tutanota.com>
>

Mime
View raw message