hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6471) Support to add min/max resource configuration for a queue
Date Sun, 14 May 2017 05:30:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-6471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16009601#comment-16009601

Wangda Tan commented on YARN-6471:

Thanks [~sunilg] for updating the patch, comments:

1) getEffectiveMaxCapacityByNormalizeUp, is there anybody use the method?

2) getEffectiveCapacityByNormalizeUp, I check the code. There're 3 places use it:
- calculateAndGetAMResourceLimitPerPartition/getUserAMResourceLimitPerPartition, we already
allow start at least 1 app if queue capacity is too low and two methods normalize_up for am-percentage.
I think it is safe to do normalize down. 
- computeUserLimit: Similarily, we allow start at least 1 container. I think it is also safe
to do normalize down.

In general, if you agree with #1/#2, can we simplify QueueQuotas and other methods? Since
we don't have to pass clusterResource around. 

Also, offline we discussed about lastClusterResource, etc. I think that is not a problem as
well since IIUC, lastClusterResource is likely to be the lastest. Please let me know if there's
any potential issues in your mind.

3) Not sure if I missed anything, it seems to me configuredResource is updated only inside
{{setupQueueConfigs}}. However if queue's configured in percentage. We may need to update
configured resources in updateClusterResource as well.

I still looking at changes which use the new abs min/max resource.

> Support to add min/max resource configuration for a queue
> ---------------------------------------------------------
>                 Key: YARN-6471
>                 URL: https://issues.apache.org/jira/browse/YARN-6471
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacity scheduler
>            Reporter: Sunil G
>            Assignee: Sunil G
>         Attachments: YARN-6471.001.patch, YARN-6471.002.patch, YARN-6471.003.patch, YARN-6471.004.patch
> This jira will track the new configurations which are needed to configure min resource
and max resource of various resource types in a queue.
> For eg: 
> {noformat}
> yarn.scheduler.capacity.root.default.memory.min-resource
> yarn.scheduler.capacity.root.default.memory.max-resource
> yarn.scheduler.capacity.root.default.vcores.min-resource
> yarn.scheduler.capacity.root.default.vcores.max-resource
> {noformat}
> Uploading a patch soon

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message