flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pa Rö <paul.roewer1...@googlemail.com>
Subject spark vs flink low memory available
Date Mon, 10 Aug 2015 13:59:04 GMT
hi community,

i have build a spark and flink k-means application.
my test case is a clustering on 1 million points on 3node cluster.

in memory bottlenecks begins flink to outsource to disk and work slowly but
works.
however spark lose executers if the memory is full and starts again
(infinety loop?).

i try to customize the memory setting with the help from the mailing list
here
<http://mail-archives.us.apache.org/mod_mbox/spark-user/201508.mbox/%3CCAH2_pyKQhFR4tBvPbt2TDHGm+ZRkCBZfNK7ueDkJPdhe472fVQ@mail.gmail.com%3E>,
thanks.
but spark not work.

is it necessary to have any configurations to be set? i mean flink work
with low memory, spark must also be able to or not?

best regards,
paul

Mime
View raw message