flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Li Peng <li.p...@doordash.com>
Subject Best way set max heap size via env variables or program arguments?
Date Tue, 31 Dec 2019 19:10:09 GMT
Hey folks, we've been running a k8 flink application, using the
taskmanager.sh script and passing in the -Djobmanager.heap.size=9000m and
-Dtaskmanager.heap.size=7000m as options to the script. I noticed from the
logs, that the Maximum heap size logged completely ignores these arguments,
and just sets the heap to the default of 922M.

I tested setting the taskmanager.heap.size and jobmanager.heap.size
manually in flink-conf.yaml, and it does work as expected (minus the heap
being set a little lower than configured). But since we want the
application to pick up different memory settings based on the environment
(local/staging/prod/etc), setting it in flink-conf isn't ideal.

So my questions are:

1. Is there actually a way to pass in the heap size via arguments to
taskmanager.sh? Is passing -Dtaskmanager.heap.size supposed to work?
2.  If not, is there a recommended way to set the heap size by environment,
like environmental variables?
3. Also, the maximum heap size logged and -Xms and -Xmx is always a little
smaller than the configured size (i.e. configuring 3000m results 2700m in
the jvm arguments, 1024m results in 922), why is that?

Thanks, and happy new year!

View raw message