flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steven Wu <stevenz...@gmail.com>
Subject help understand/debug high memory footprint on jobmanager
Date Thu, 28 Jun 2018 23:29:43 GMT
First, some context about the job
* embarrassingly parallel: all operators are chained together
* parallelism is over 1,000
* stateless except for Kafka source operators. checkpoint size is 8.4 MB.
* set "state.backend.fs.memory-threshold" so that only jobmanager writes to
S3 to checkpoint
* internal checkpoint with 10 checkpoints retained in history

We don't expect jobmanager to use much memory at all. But it seems that
this high memory footprint (or leak) happened occasionally, maybe under
certain conditions. Any hypothesis?


41,567 ExecutionVertex objects retained 9+ GB of memory

Expanded in one ExecutionVertex. it seems to storing the kafka offsets for
source operator

View raw message