flink-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From fhue...@apache.org
Subject flink git commit: [FLINK-3621] [docs] Improve documentation of memory configuration parameters
Date Wed, 16 Mar 2016 13:18:44 GMT
Repository: flink
Updated Branches:
  refs/heads/release-1.0 165065c3e -> 6749874dc


[FLINK-3621] [docs] Improve documentation of memory configuration parameters

This closes #1801


Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/6749874d
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/6749874d
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/6749874d

Branch: refs/heads/release-1.0
Commit: 6749874dccdce06d9ac7180e471b159d84d82eaa
Parents: 165065c
Author: Fabian Hueske <fhueske@apache.org>
Authored: Wed Mar 16 13:58:39 2016 +0100
Committer: Fabian Hueske <fhueske@apache.org>
Committed: Wed Mar 16 14:18:24 2016 +0100

----------------------------------------------------------------------
 docs/setup/config.md | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/flink/blob/6749874d/docs/setup/config.md
----------------------------------------------------------------------
diff --git a/docs/setup/config.md b/docs/setup/config.md
index 70963bb..28b1429 100644
--- a/docs/setup/config.md
+++ b/docs/setup/config.md
@@ -67,13 +67,13 @@ without explicit scheme definition, such as `/user/USERNAME/in.txt`, is
going to
 
 ### Managed Memory
 
-By default, Flink allocates a fraction of 0.7 of the total memory configured via `taskmanager.heap.mb`
for its managed memory. Managed memory helps Flink to run the operators efficiently. It prevents
OutOfMemoryExceptions because Flink knows how much memory it can use to execute operations.
If Flink runs out of managed memory, it utilizes disk space. Using managed memory, some operations
can be performed directly on the raw data without having to deserialize the data to convert
it into Java objects. All in all, managed memory improves the robustness and speed of the
system.
+By default, Flink allocates a fraction of `0.7` of the total memory configured via `taskmanager.heap.mb`
for its managed memory. Managed memory helps Flink to run the operators efficiently. It prevents
`OutOfMemoryException`s because Flink knows how much memory it can use to execute operations.
If Flink runs out of managed memory, it utilizes disk space. Using managed memory, some operations
can be performed directly on the raw data without having to deserialize the data to convert
it into Java objects. All in all, managed memory improves the robustness and speed of the
system.
 
 The default fraction for managed memory can be adjusted using the `taskmanager.memory.fraction`
parameter. An absolute value may be set using `taskmanager.memory.size` (overrides the fraction
parameter). If desired, the managed memory may be allocated outside the JVM heap. This may
improve performance in setups with large memory sizes.
 
-- `taskmanager.memory.size`: The amount of memory (in megabytes) that the task manager reserves
on the JVM's heap space for sorting, hash tables, and caching of intermediate results. If
unspecified (-1), the memory manager will take a fixed ratio of the heap memory available
to the JVM, as specified by `taskmanager.memory.fraction`. (DEFAULT: -1)
+- `taskmanager.memory.size`: The amount of memory (in megabytes) that the task manager reserves
on-heap or off-heap (depending on `taskmanager.memory.off-heap`) for sorting, hash tables,
and caching of intermediate results. If unspecified (-1), the memory manager will take a fixed
ratio with respect to the size of the task manager JVM as specified by `taskmanager.memory.fraction`.
(DEFAULT: -1)
 
-- `taskmanager.memory.fraction`: The relative amount of memory that the task manager reserves
for sorting, hash tables, and caching of intermediate results. For example, a value of 0.8
means that TaskManagers reserve 80% of the JVM's heap space for internal data buffers, leaving
20% of the JVM's heap space free for objects created by user-defined functions. (DEFAULT:
0.7) This parameter is only evaluated, if `taskmanager.memory.size` is not set.
+- `taskmanager.memory.fraction`: The relative amount of memory (with respect to `taskmanager.heap.mb`)
that the task manager reserves for sorting, hash tables, and caching of intermediate results.
For example, a value of `0.8` means that a task manager reserves 80% of its memory (on-heap
or off-heap depending on `taskmanager.memory.off-heap`) for internal data buffers, leaving
20% of free memory for the task manager's heap for objects created by user-defined functions.
(DEFAULT: 0.7) This parameter is only evaluated, if `taskmanager.memory.size` is not set.
 
 - `taskmanager.memory.off-heap`: If set to `true`, the task manager allocates memory which
is used for sorting, hash tables, and caching of intermediate results outside of the JVM heap.
For setups with larger quantities of memory, this can improve the efficiency of the operations
performed on the memory (DEFAULT: false).
 


Mime
View raw message