flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shuai-xu <...@git.apache.org>
Subject [GitHub] flink pull request #4991: [FLINK-7928] [runtime] extend the resources in Res...
Date Fri, 08 Dec 2017 05:59:43 GMT
Github user shuai-xu commented on a diff in the pull request:

    --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/types/ResourceProfile.java
    @@ -61,16 +80,26 @@
     	 * @param heapMemoryInMB The size of the heap memory, in megabytes.
     	 * @param directMemoryInMB The size of the direct memory, in megabytes.
     	 * @param nativeMemoryInMB The size of the native memory, in megabytes.
    +	 * @param memoryForInputInMB The size of the memory for input, in megabytes.
    +	 * @param memoryForOutputInMB The size of the memory for output, in megabytes.
     	public ResourceProfile(
     			double cpuCores,
     			int heapMemoryInMB,
     			int directMemoryInMB,
    -			int nativeMemoryInMB) {
    +			int nativeMemoryInMB,
    +			int memoryForInputInMB,
    +			int memoryForOutputInMB,
    --- End diff --
    I think resource spec contains the resource user need to run their code, while resource
profile contains the resource for running a task. So resource profile should also contain
the part of resource used by flink system. We divide these part of resource to memoryForInputInMB
and memoryForOutputInMB, and separate them from heap memory and  direct memory so as to different
resource managers can choose different strategies. For example, per job resource manager need
all these resource when allocating a task manager. but session manager may not consider the
memoryForInputInMB and memoryForOutputInMB when assign a slot, as these part is decide when
the session cluster is created. Do you think it make sense?


View raw message