hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "zhengchenyu (JIRA)" <j...@apache.org>
Subject [jira] [Issue Comment Deleted] (YARN-4090) Make Collections.sort() more efficient in FSParentQueue.java
Date Fri, 25 Nov 2016 10:56:58 GMT

     [ https://issues.apache.org/jira/browse/YARN-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

zhengchenyu updated YARN-4090:
    Comment: was deleted

(was: here we see a dead block: 
"IPC Server handler 98 on 8032" is waiting for lock (0x00007f42e17a5ed8)
"IPC Server handler 76 on 8032" got the lock (0x00007f42e17a5ed8), is is waiting for lock
"ResourceManager Event Processor" got the lock (0x00007f42df3e8450),is waiting for lock (0x00007f42e17a5ed8)

In fact, 0x00007f42e17a5ed8 is a object lock of FSParentQueue, here I called this root.Parent.
0x00007f42df3e8450 is another object lock of FSParentQueue, this is the child queue object
of 0x00007f42e17a5ed8. here I called this root.Parent.

Let's trace these thread.
(1) ResourceManager Event Processor
         //got the lock 0x00007f42e0c7cf50				
	   //got the lock 0x00007f42df3e8450 which is the object lock of root.Parent.Child
	     //wait for 0x00007f42e17a5ed8 which is the object lock of root.Parent
(2) IPC Server handler 76 on 8032
     //got the lock 0x00007f42e17a5ed8
       //got the lock 0x00007f42df3e8450
The left thread is unnecessary to analyse. Here we can see decResourceUsage got the object
lock from bottom to top, but getQueueUserAcls got the object lock from top to bottom.getQueueUserAcls
got the object lock of root and root.Parent, and waits for root.Parent.Child. But decResourceUsage
got the object lock of root.Parent.Child, and waits for root.Parnt. That's a deadlock.
I recommend that decResourceUsage is rewriten with the way of getting the object lock from
top to bottom.)

> Make Collections.sort() more efficient in FSParentQueue.java
> ------------------------------------------------------------
>                 Key: YARN-4090
>                 URL: https://issues.apache.org/jira/browse/YARN-4090
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: fairscheduler
>            Reporter: Xianyin Xin
>            Assignee: Xianyin Xin
>         Attachments: YARN-4090-TestResult.pdf, YARN-4090-preview.patch, YARN-4090.001.patch,
YARN-4090.002.patch, YARN-4090.003.patch, sampling1.jpg, sampling2.jpg
> Collections.sort() consumes too much time in a scheduling round.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message