hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Miklos Szegedi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6384) Add configuration property to set max CPU usage when strict-resource-usage is false with cgroups
Date Fri, 19 Jan 2018 17:44:01 GMT

    [ https://issues.apache.org/jira/browse/YARN-6384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16332616#comment-16332616
] 

Miklos Szegedi commented on YARN-6384:
--------------------------------------

Thank you for the latest patch [~lolee_k] and for the review [~templedf]. It looks good in
general, I have two comments.
{code:java}
102	if (this.maxResourceUsagePercentInStrictMode <= 0.0f) {
103	this.maxResourceUsagePercentInStrictMode =
104	YarnConfiguration.DEFAULT_NM_LINUX_CONTAINER_CGROUPS_MAX_RESOURCE_USAGE_PERCENT;
105	}{code}
It would be nice to give a warning log here that the setting was invalid and we reset to
the default.

Also, even though the new configuration is named max-resource-usage-percent, it only applies
to CPU for now. I think this is good, it may prove future proof, however it would be nice
to file a new Jira that will handle memory and other subsystems.

> Add configuration property to set max CPU usage when strict-resource-usage is false with
cgroups
> ------------------------------------------------------------------------------------------------
>
>                 Key: YARN-6384
>                 URL: https://issues.apache.org/jira/browse/YARN-6384
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: dengkai
>            Assignee: dengkai
>            Priority: Major
>         Attachments: YARN-6384-0.patch, YARN-6384-1.patch, YARN-6384-2.patch, YARN-6384-3.patch,
YARN-6384-4.patch, YARN-6384-5.patch
>
>
> When using cgroups on yarn, if yarn.nodemanager.linux-container-executor.cgroups.strict-resource-usage
is false, user may get very more cpu time than expected based on the vcores. There should
be a upper limit even resource-usage is not strict, just like a percentage which user can
get more than promised by vcores. I think it's important in a shared cluster.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message