flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From yunfan123 <yunfanfight...@foxmail.com>
Subject Flink takes too much memory in record serializer.
Date Tue, 14 Nov 2017 10:11:24 GMT
In the class org.apache.flink.runtime.io.network.api.writer.RecordWriter, it
has same number of serializers with the numChannels.
If I first operator has 500 parallels and the next operator has 1000
And every message in flink is 2MB.
The job takes 500 * 1000 * 2MB as 1TB memory in totally!!!
Can I do anything to reduce the memory usage.

Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

View raw message