flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From NicoK <...@git.apache.org>
Subject [GitHub] flink pull request #4594: [FLINK-7517][network] let NettyBufferPool extend P...
Date Thu, 09 Nov 2017 08:15:38 GMT
Github user NicoK commented on a diff in the pull request:

    https://github.com/apache/flink/pull/4594#discussion_r149889426
  
    --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/NettyBufferPool.java
---
    @@ -52,51 +48,61 @@
     	/** Configured chunk size for the arenas. */
     	private final int chunkSize;
     
    +	/** We strictly prefer direct buffers and disallow heap allocations. */
    +	private static final boolean PREFER_DIRECT = true;
    +
    +	/**
    +	 * Arenas allocate chunks of pageSize << maxOrder bytes. With these defaults,
this results in
    +	 * chunks of 16 MB.
    +	 *
    +	 * @see #MAX_ORDER
    +	 */
    +	private static final int PAGE_SIZE = 8192;
    +
    +	/**
    +	 * Arenas allocate chunks of pageSize << maxOrder bytes. With these defaults,
this results in
    +	 * chunks of 16 MB.
    +	 *
    +	 * @see #PAGE_SIZE
    +	 */
    +	private static final int MAX_ORDER = 11;
    +
     	/**
     	 * Creates Netty's buffer pool with the specified number of direct arenas.
     	 *
     	 * @param numberOfArenas Number of arenas (recommended: 2 * number of task
     	 *                       slots)
     	 */
     	public NettyBufferPool(int numberOfArenas) {
    +		super(
    +			PREFER_DIRECT,
    +			// No heap arenas, please.
    +			0,
    +			// Number of direct arenas. Each arena allocates a chunk of 16 MB, i.e.
    +			// we allocate numDirectArenas * 16 MB of direct memory. This can grow
    +			// to multiple chunks per arena during runtime, but this should only
    +			// happen with a large amount of connections per task manager. We
    +			// control the memory allocations with low/high watermarks when writing
    +			// to the TCP channels. Chunks are allocated lazily.
    +			numberOfArenas,
    +			PAGE_SIZE,
    +			MAX_ORDER);
    +
     		checkArgument(numberOfArenas >= 1, "Number of arenas");
    --- End diff --
    
    Yes, it would be nice to be able to do so but since this is the constructor, it is not
possible. I guess that `super()` may fail itself with an invalid parameter - if not, we fail
a bit afterwards.


---

Mime
View raw message