hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jian He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3139) Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler
Date Wed, 21 Sep 2016 09:49:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509412#comment-15509412
] 

Jian He commented on YARN-3139:
-------------------------------

- Is removing these locks  ok?
{code}
  // It is crucial to acquire leaf queue lock first to prevent:
  // 1. Race condition when calculating the delta resource in
  //    SchedContainerChangeRequest
  // 2. Deadlock with the scheduling thread.
{code}
-  Also, here.. the queue lock is removed
{code}
    synchronized(leafQueue) { 
      SchedulerNode schedulerNode =
          getSchedulerNode(rmContainer.getAllocatedNode());
      SchedContainerChangeRequest decreaseRequest =
          new SchedContainerChangeRequest(this.rmContext, schedulerNode,
              rmContainer, rmContainer.getLastConfirmedResource());
      decreaseContainer(decreaseRequest, application);
    }
{code}
- In fairscheduler,  here, locking on application is changed to locking on scheduler ?
{code}
try {
      readLock.lock();
      if (!ask.isEmpty()) {
        if (LOG.isDebugEnabled()) {
          LOG.debug(
              "allocate: pre-update" + " applicationAttemptId=" + appAttemptId
                  + " application=" + application.getApplicationId());
        }
        application.showRequests();
{code}
- the lock is outside try block...
{code}
      // Commit the reload; also create any queue defined in the alloc file
      // if it does not already exist, so it can be displayed on the web UI.

      writeLock.lock();
      try {
{code}


> Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler
> ----------------------------------------------------------------------
>
>                 Key: YARN-3139
>                 URL: https://issues.apache.org/jira/browse/YARN-3139
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager, scheduler
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-3139.0.patch, YARN-3139.1.patch
>
>
> Enhance locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler, as mentioned
in YARN-3091, a possible solution is using read/write lock. Other fine-graind locks for specific
purposes / bugs should be addressed in separated tickets.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message