flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@apache.org>
Subject Re: Naive question
Date Sun, 19 Oct 2014 19:50:26 GMT
In fact, the use-cases of Spark and Flink overlap a bit. However, the
technology used under the hood is quite different. Flink shares a lot of
similarities with relational DBMS. Data is serialized in byte buffers
and processed a lot in binary representation. This also allows for
fine-grained memory control. Flink uses a pipelined processing model and it
has a cost-based optimizer that selects execution strategies and avoids
expensive partitioning and sorting steps. Moreover, Flink features a
special kind of iterations (delta-iterations) that can significantly reduce
the amount of computations as iterations go on (the vertex-centric
computing model of Pregel / Giraph is a special kind of that).

Btw. The academic projects from which Spark and Flink originated
started about the same time ;-)

Best, Fabian

2014-10-19 20:33 GMT+02:00 Mohit Singh <mohit1007@gmail.com>:

> Hi,
>   Good to see another interesting apache incubator project but I am just
> curious on what is flink trying to solve which spark is not currently
> addressing? I am sure you get this question a lot.. :)
> Thanks
> --
> Mohit
> "When you want success as badly as you want the air, then you will get it.
> There is no other secret of success."
> -Socrates

View raw message