flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paulo Cezar <paulo.ce...@gogeo.io>
Subject Re: High virtual memory usage
Date Mon, 19 Dec 2016 11:32:04 GMT
  - Are you using RocksDB?


  - What is your flink configuration, especially around memory settings?

I'm using default config with 2GB for jobmanager and 5GB for taskmanagers.
I'm starting flink via "./bin/yarn-session.sh -d -n 5 -jm 2048 -tm 5120 -s
4 -nm 'Flink'"

  - What do you use for TaskManager heap size? Any manual value, or do you
let Flink/Yarn set it automatically based on container size?

No manual values here. YARN config is pretty much default with maximum
allocation of 12GB of physical memory and ratio between virtual memory to
physical memory 2.1 (via yarn.nodemanager.vmem-pmem-ratio).

  - Do you use any libraries or connectors in your program?

I'm using  flink-connector-kafka-0.10_2.11, a MongoDB client, a gRPC client
and some http libraries like unirest and Apache HttpClient.

  - Also, can you tell us what OS you are running on?

My YARN cluster runs on Docker containers (docker version 1.12) with images
based on Ubuntu 14.04. Host OS is Ubuntu 14.04.4 LTS (GNU/Linux
3.19.0-65-generic x86_64).

View raw message