hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haibo Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4511) Common scheduler changes supporting scheduler-specific implementations
Date Fri, 20 Oct 2017 18:06:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16212971#comment-16212971
] 

Haibo Chen commented on YARN-4511:
----------------------------------

bq. however we need to make sure it reflects the state of the object, so for example allocateContainer()
should set this value as the last step after the allocatedContainers.put() call. 
bq. containerResourceReleased should decrease resourceAllocatedPendingLaunch, if the container
has not been started, yet.
Good points, will address in the next patch.
bq. I think that it would be nicer to lock around these two calls to become atomic.
swapContainer() is already protected in a writeLock, so it is already atomic, no?

bq. isValidGuaranteedContainer and isValidOpportunisticContainer contain the same code. Should
they be different? 
I'm inclined to keep both of them. The caller may want to check whether it is a guaranteed
or opportunistic, not just whether it has been allocated on the node
It just so happens that we are sharing the same map for both OPPORTUNISTIC and GUARANTEED
containers, hence the code is identical.
I'll add Execution Type check to be more rigorous.

bq. allocatedContainers.remove(containerId); can be placed outside the if.
{code:java}
if (container.getExecutionType() == ExecutionType.GUARANTEED) {
      guaranteedContainerResourceReleased(container);
      numGuaranteedContainers--;
} else {
      opportunisticContainerResourceReleased(container);
      numOpportunisticContainers--;
}
allocatedContainers.remove(containerId);
{code}
The above code will update the num*Containers counter before allocatedContainers is updated,
so I think we should keep it as it.

> Common scheduler changes supporting scheduler-specific implementations
> ----------------------------------------------------------------------
>
>                 Key: YARN-4511
>                 URL: https://issues.apache.org/jira/browse/YARN-4511
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Wangda Tan
>            Assignee: Haibo Chen
>         Attachments: YARN-4511-YARN-1011.00.patch, YARN-4511-YARN-1011.01.patch, YARN-4511-YARN-1011.02.patch,
YARN-4511-YARN-1011.03.patch, YARN-4511-YARN-1011.04.patch, YARN-4511-YARN-1011.05.patch,
YARN-4511-YARN-1011.06.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message