flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shashwat Rastogi <shashwat.rast...@reflektion.com>
Subject Can someone explain memory usage in a flink worker?
Date Thu, 27 Jul 2017 09:29:05 GMT
Hi,

I have a setup of 7 task managers, each with 64GB of physical memory out of which I have allocated
35GB as task manager’s heap memory. I am using rocksdb as state backend.

I see a lot of anomalies in the reports generated by the Flink-UI vs my system metrics. Can
someone please help explaining what is happening here.

- Why does the flink-ui shows 35 GB as free memory when the system has currently occupied
48.4 GB of memory, which leaves only (62.5-48.4)14.1 GB free. 
- Where is the memory used by RocksDb displayed? The machine does not do anything except serving
as the flink worker, so can I assume that 62.5 GB - 35 GB - 14.1 Gb - 523MB = 12.9 GB is the
memory used by RocksDb? When I do `ps -ef | grep rocksdb` I don’t see any process running,
is this normal?
- Also, my system metrics shows that the memory usage keeps on increasing until the task-manager
itself gets killed <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Memory-Leak-Flink-RocksDB-td14439.html>
but when I see in the Flink-Ui I always see lot of free memory. I am using the default configuration
so I don’t think flink managed memory occupies non-heap memory. I am not able to figure
out where does this ever-increasing memory consumption is coming from, my guess is this is
used by RocksDb.

FLINK-UI




SYSTEM METRICS




Thanks in advance.
Shashwat
Mime
View raw message