flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: taskmanager memory leak
Date Wed, 20 Jul 2016 18:52:22 GMT
Hi!

In order to answer this, we need a bit more information. Here are some
followup questions:

  - Did you submit any job to the cluster, or is the memory just growing
even on an idle TaskManager?
  - If you are running a job, do you use the RocksDB state backend, of the
FileSystem state backend?
  - Does it grow infinitely, or simply up a certain point and then goes
down again?

Greetings,
Stephan


On Wed, Jul 20, 2016 at 5:58 PM, 김동일 <kim.same@gmail.com> wrote:

> oh. my flink version is 1.0.3.
>
>
> ---------- Forwarded message ----------
> From: 김동일 <kim.same@gmail.com>
> Date: Thu, Jul 21, 2016 at 12:52 AM
> Subject: taskmanager memory leak
> To: user@flink.apache.org
>
>
> I've set up cluster(stand alone).
> Taskmanager consumes memory over the Xmx property and it grows up
> continuously.
> I saw this link(
> http://mail-archives.apache.org/mod_mbox/flink-dev/201606.mbox/%3CCAK2vtervsw4muBOc4SWix0mR6Y9biJznjuYpF6_f9f0g9-_6LA@mail.gmail.com%3E
> ).
> So i set the taskmanager.memory.preallocation value to true but that is
> not solution.
>
> my java version is
>
> java version "1.8.0_20"
> Java(TM) SE Runtime Environment (build 1.8.0_20-b26)
> Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode)
>
> and my flink-conf.yaml
> <http://mail-archives.apache.org/mod_mbox/flink-dev/201606.mbox/%3CCAK2vtervsw4muBOc4SWix0mR6Y9biJznjuYpF6_f9f0g9-_6LA@mail.gmail.com%3E>
> <http://mail-archives.apache.org/mod_mbox/flink-dev/201606.mbox/%3CCAK2vtervsw4muBOc4SWix0mR6Y9biJznjuYpF6_f9f0g9-_6LA@mail.gmail.com%3E>
> <http://mail-archives.apache.org/mod_mbox/flink-dev/201606.mbox/%3CCAK2vtervsw4muBOc4SWix0mR6Y9biJznjuYpF6_f9f0g9-_6LA@mail.gmail.com%3E>
> <http://mail-archives.apache.org/mod_mbox/flink-dev/201606.mbox/%3CCAK2vtervsw4muBOc4SWix0mR6Y9biJznjuYpF6_f9f0g9-_6LA@mail.gmail.com%3E>
> <http://mail-archives.apache.org/mod_mbox/flink-dev/201606.mbox/%3CCAK2vtervsw4muBOc4SWix0mR6Y9biJznjuYpF6_f9f0g9-_6LA@mail.gmail.com%3E>
> <http://mail-archives.apache.org/mod_mbox/flink-dev/201606.mbox/%3CCAK2vtervsw4muBOc4SWix0mR6Y9biJznjuYpF6_f9f0g9-_6LA@mail.gmail.com%3E>
> <http://mail-archives.apache.org/mod_mbox/flink-dev/201606.mbox/%3CCAK2vtervsw4muBOc4SWix0mR6Y9biJznjuYpF6_f9f0g9-_6LA@mail.gmail.com%3E>
> <http://mail-archives.apache.org/mod_mbox/flink-dev/201606.mbox/%3CCAK2vtervsw4muBOc4SWix0mR6Y9biJznjuYpF6_f9f0g9-_6LA@mail.gmail.com%3E>
> <http://mail-archives.apache.org/mod_mbox/flink-dev/201606.mbox/%3CCAK2vtervsw4muBOc4SWix0mR6Y9biJznjuYpF6_f9f0g9-_6LA@mail.gmail.com%3E>
> <http://mail-archives.apache.org/mod_mbox/flink-dev/201606.mbox/%3CCAK2vtervsw4muBOc4SWix0mR6Y9biJznjuYpF6_f9f0g9-_6LA@mail.gmail.com%3E>
> <http://mail-archives.apache.org/mod_mbox/flink-dev/201606.mbox/%3CCAK2vtervsw4muBOc4SWix0mR6Y9biJznjuYpF6_f9f0g9-_6LA@mail.gmail.com%3E>
> <http://mail-archives.apache.org/mod_mbox/flink-dev/201606.mbox/%3CCAK2vtervsw4muBOc4SWix0mR6Y9biJznjuYpF6_f9f0g9-_6LA@mail.gmail.com%3E>
> <http://mail-archives.apache.org/mod_mbox/flink-dev/201606.mbox/%3CCAK2vtervsw4muBOc4SWix0mR6Y9biJznjuYpF6_f9f0g9-_6LA@mail.gmail.com%3E>
> <http://mail-archives.apache.org/mod_mbox/flink-dev/201606.mbox/%3CCAK2vtervsw4muBOc4SWix0mR6Y9biJznjuYpF6_f9f0g9-_6LA@mail.gmail.com%3E>
> <http://mail-archives.apache.org/mod_mbox/flink-dev/201606.mbox/%3CCAK2vtervsw4muBOc4SWix0mR6Y9biJznjuYpF6_f9f0g9-_6LA@mail.gmail.com%3E>
> <http://mail-archives.apache.org/mod_mbox/flink-dev/201606.mbox/%3CCAK2vtervsw4muBOc4SWix0mR6Y9biJznjuYpF6_f9f0g9-_6LA@mail.gmail.com%3E>
> <http://mail-archives.apache.org/mod_mbox/flink-dev/201606.mbox/%3CCAK2vtervsw4muBOc4SWix0mR6Y9biJznjuYpF6_f9f0g9-_6LA@mail.gmail.com%3E>
>
> env.java.home: /usr/java/default
> jobmanager.rpc.address: internal.stream01.denma.ggportal.net
> jobmanager.rpc.port: 6123
> jobmanager.heap.mb: 2048
> taskmanager.heap.mb: 8192
> taskmanager.memory.off-heap: true
> taskmanager.numberOfTaskSlots: 4
> taskmanager.memory.preallocate: false
> parallelism.default: 2
> jobmanager.web.port: 8081
> jobmanager.web.submit.enable: true
> state.backend: filesystem
> state.backend.fs.checkpointdir: s3a://denma.live/flink/datum/checkpoints
> taskmanager.network.numberOfBuffers: 8192
> taskmanager.tmp.dirs: /opt/flink/var/tmp
> fs.hdfs.hadoopconf: /opt/flink/conf/
> recovery.mode: zookeeper
> recovery.zookeeper.quorum: ....
> recovery.zookeeper.storageDir: s3a://denma.live/flink/datum/recovery
> recovery.jobmanager.port: 50000-50100
> recovery.zookeeper.path.root: /flink
> blob.server.port: 50100-50200
> blob.storage.directory: /opt/flink/var/tmp/flink-blob
> taskmanager.rpc.port: 6122
> taskmanager.data.port: 6121
>
>
> i need help. what shall i do?
> thx in advance.
>
>
>
> --
> <A HREF="http://www.kiva.org" TARGET="_top">
> <IMG SRC="http://www.kiva.org/images/bannerlong.png" WIDTH="460"
> HEIGHT="60" ALT="Kiva - loans that change lives" BORDER="0"
> ALIGN="BOTTOM"></A>
>

Mime
View raw message