flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kien Truong <duckientru...@gmail.com>
Subject Re: Flink memory usage
Date Sat, 04 Nov 2017 12:56:03 GMT
Hi,

How did you measure the memory usage ?

JVM processes tend to occupy the maximum memory allocated to them, 
regardless of whether those memory are actively in used or not. To 
correctly measure the memory usage, you should use Flink's metric system[1]

Regards,

Kien

[1] 
https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/metrics.html#system-metrics

On 11/4/2017 3:15 AM, AndreaKinn wrote:
> Hi,
> I would like to share some considerations about Flink memory consumption.
> I have a cluster composed of three nodes: 1 used both as JM and TM and other
> 2 TM.
>
> I ran two identical applications (in different moments) on it. The only
> difference is that on the second one I doubled every operators, essentially
> to check what changes in resource's usage.
>
> Analysing the outcomes on cpu side effectively the efforts are doubled.
> Doing the same with memory I had these results:
>
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/t985/Screen_Shot_2017-11-03_at_21.png>
>
> which to me seems completely counterintuitive since the results are
> essentially equal.
> I can imagine in the second case the memory was effectively almost full but
> why Flink gets such a lot of memory even in the first case?
> How it is explained this behaviour?
>
>
>
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Mime
View raw message