flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ovidiu-Cristian MARCU <ovidiu-cristian.ma...@inria.fr>
Subject Re: off-heap size feature request
Date Wed, 16 Mar 2016 11:40:07 GMT
I will try this one: taskmanager.memory.size. So I should expect this will be the off-heap
memory size, right?
I am using taskmanager.heap.mb=some value, taskmanager.memory.off-heap: true
Memory usage goes up to 99%.

The documentation is confusing:

taskmanager.memory.size: The amount of memory (in megabytes) that the task manager reserves
on the JVM’s heap space for sorting, hash tables, and caching of intermediate results. If
unspecified (-1), the memory manager will take a fixed ratio of the heap memory available
to the JVM, as specified by taskmanager.memory.fraction.  [1]

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.0/setup/config.html#managed-memory


> On 16 Mar 2016, at 12:13, Fabian Hueske <fhueske@gmail.com> wrote:
> Hi Ovidiu,
> the parameters to configure the amount of managed memory (taskmanager.memory.size, taskmanager.memory.fraction)
are valid for on and off-heap memory. 
> Have you tried these parameters and didn't they work as expected?
> Best, Fabian
> 2016-03-16 11:43 GMT+01:00 Ovidiu-Cristian MARCU <ovidiu-cristian.marcu@inria.fr <mailto:ovidiu-cristian.marcu@inria.fr>>:
> Hi,
> Is it possible to add a parameter off-heap.size for the task manager off-heap memory
> It is not possible to limit the off-heap memory size, at least I found nothing in the
> [1] https://ci.apache.org/projects/flink/flink-docs-release-1.0/setup/config.html#managed-memory
> Best,
> Ovidiu

View raw message