Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Tue, 25 Oct 2016 14:48:59 +0000 (UTC)
From: "Varun Vasudev (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.13001323.1472566042000.79655.1477406939076@Atlassian.JIRA>
In-Reply-To: <JIRA.13001323.1472566042000@Atlassian.JIRA>
References: <JIRA.13001323.1472566042000@Atlassian.JIRA> <JIRA.13001323.1472566042959@arcas>
Subject: [jira] [Commented] (YARN-5587) Add support for resource profiles
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 25 Oct 2016 14:49:01 -0000


    [ https://issues.apache.org/jira/browse/YARN-5587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15605509#comment-15605509 ] 

Varun Vasudev commented on YARN-5587:
-------------------------------------

bq. In Resources, you moved the Suppress deprecation warning from the setMemorySize(long) method to the setMemory(int). Was that intentional ?

Yes - setMemory is deprecated not setMemorySize. Findbugs complains if I don't supress the warning.

{quote}
    AMRMClient::ContainerRequest : Wondering if we need to allow a Container request to specify both a profile name and a Resource (capability). If they do specify both, what does that mean ?
    Similarly, in the RemoteRequestTable, the RR should be keyed using the Resource (capability) derived from the profileName.
{quote}

Good question - my opinion is that profile + capability means take the profile and override it with the capability. So for example if I say I want profile 'large' with capability <4096M, 2 cores> - that means take all the values from profile large but use 4096M for memory and 2 cores for cpu. This leads to a follow up question - what if you ask for profile 'large' with capability <4096M, 0 cores>(this will happen because the Resource object doesn't have 'null' values for memory and vcores)? In this case my proposal is to only use non-zero overrides.

The reason I went with this approach is that most users are used to running spark and MR jobs with memory and cores specified. They'll continue to run these applications the same way. This approach allows an easy way for admins to turn on resource profiles without affecting users. However I also accept that the 'only consider non-zero' values for overrides might seem hackish - I'm absolutely open to alternatives.

> Add support for resource profiles
> ---------------------------------
>
>                 Key: YARN-5587
>                 URL: https://issues.apache.org/jira/browse/YARN-5587
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Varun Vasudev
>            Assignee: Varun Vasudev
>         Attachments: YARN-5587-YARN-3926.001.patch, YARN-5587-YARN-3926.002.patch, YARN-5587-YARN-3926.003.patch, YARN-5587-YARN-3926.004.patch, YARN-5587-YARN-3926.005.patch
>
>
> Add support for resource profiles on the RM side to allow users to use shorthands to specify resource requirements.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org