Mailing-List: contact issues-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
From: shuai-xu <git@git.apache.org>
To: issues@flink.incubator.apache.org
Reply-To: issues@flink.incubator.apache.org
References: <git-pr-4991-flink@git.apache.org>
In-Reply-To: <git-pr-4991-flink@git.apache.org>
Subject: [GitHub] flink pull request #4991: [FLINK-7928] [runtime] extend the resources in Res...
Content-Type: text/plain
Message-Id: <20171208055943.68BF0F615F@git1-us-west.apache.org>
Date: Fri,  8 Dec 2017 05:59:43 +0000 (UTC)
archived-at: Fri, 08 Dec 2017 05:59:47 -0000

Github user shuai-xu commented on a diff in the pull request:

    https://github.com/apache/flink/pull/4991#discussion_r155708446
  
    --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/types/ResourceProfile.java ---
    @@ -61,16 +80,26 @@
     	 * @param heapMemoryInMB The size of the heap memory, in megabytes.
     	 * @param directMemoryInMB The size of the direct memory, in megabytes.
     	 * @param nativeMemoryInMB The size of the native memory, in megabytes.
    +	 * @param memoryForInputInMB The size of the memory for input, in megabytes.
    +	 * @param memoryForOutputInMB The size of the memory for output, in megabytes.
     	 */
     	public ResourceProfile(
     			double cpuCores,
     			int heapMemoryInMB,
     			int directMemoryInMB,
    -			int nativeMemoryInMB) {
    +			int nativeMemoryInMB,
    +			int memoryForInputInMB,
    +			int memoryForOutputInMB,
    --- End diff --
    
    I think resource spec contains the resource user need to run their code, while resource profile contains the resource for running a task. So resource profile should also contain the part of resource used by flink system. We divide these part of resource to memoryForInputInMB and memoryForOutputInMB, and separate them from heap memory and  direct memory so as to different resource managers can choose different strategies. For example, per job resource manager need all these resource when allocating a task manager. but session manager may not consider the memoryForInputInMB and memoryForOutputInMB when assign a slot, as these part is decide when the session cluster is created. Do you think it make sense?


---