flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Li Peng <li.p...@doordash.com>
Subject Best way set max heap size via env variables or program arguments?
Date Tue, 31 Dec 2019 19:10:09 GMT
Hey folks, we've been running a k8 flink application, using the
taskmanager.sh script and passing in the -Djobmanager.heap.size=9000m and
-Dtaskmanager.heap.size=7000m as options to the script. I noticed from the
logs, that the Maximum heap size logged completely ignores these arguments,
and just sets the heap to the default of 922M.

I tested setting the taskmanager.heap.size and jobmanager.heap.size
manually in flink-conf.yaml, and it does work as expected (minus the heap
being set a little lower than configured). But since we want the
application to pick up different memory settings based on the environment
(local/staging/prod/etc), setting it in flink-conf isn't ideal.

So my questions are:

1. Is there actually a way to pass in the heap size via arguments to
taskmanager.sh? Is passing -Dtaskmanager.heap.size supposed to work?
2.  If not, is there a recommended way to set the heap size by environment,
like environmental variables?
3. Also, the maximum heap size logged and -Xms and -Xmx is always a little
smaller than the configured size (i.e. configuring 3000m results 2700m in
the jvm arguments, 1024m results in 922), why is that?

Thanks, and happy new year!
Li

Mime
View raw message