flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefan Richter <s.rich...@data-artisans.com>
Subject Re: org.apache.flink.runtime.io.network.NetworkEnvironment causing memory leak?
Date Thu, 16 Nov 2017 09:35:29 GMT

I cannot spot anything that indicates a leak from your screenshots. Maybe you misinterpret
the numbers? In your heap dump, there is only a single instance of org.apache.flink.runtime.io.network.NetworkEnvironment
and it retains about 400,000,000 bytes from being GCed because it holds references to the
network buffers. This is perfectly normal because this the buffer pool is part of this object,
and for as long as it lives, the referenced buffers should not be GCed and the current size
of all your buffers is around 400 million bytes.

Your heap space is also not growing without bounds, but always goes down after a GC was performed.
Looks fine to me.

Last, I think the number of G1_Young_Generation is a counter of how many gc cycles have been
performed and the time is a sum. So naturally, those values would always increase.


> Am 15.11.2017 um 18:35 schrieb Hao Sun <hasun@zendesk.com>:
> Hi team, I am looking at some memory/GC issues for my flink setup. I am running flink
1.3.2 in docker for my development environment. Using Kubernetes for production.
> I see instances of org.apache.flink.runtime.io.network.NetworkEnvironment are increasing
dramatically and not GC-ed very well for my application.
> My simple app collects Kafka events and transforms the information and logs the results
> Is this expected? I am new to Java memory analysis not sure what is actually wrong.
> <image.png>
> <image.png>
> <image.png>
> <image.png>

View raw message