hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3762) FairScheduler: CME on FSParentQueue#getQueueUserAclInfo
Date Wed, 03 Jun 2015 19:03:38 GMT

    [ https://issues.apache.org/jira/browse/YARN-3762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14571502#comment-14571502
] 

Karthik Kambatla commented on YARN-3762:
----------------------------------------

Thanks for the review, Arun. Good points.

bq. what happens if the collection is modified in between..
The two possible modifications are adding/removing a child queue. Adding a child queue to
the end of the list doesn't affect container assignment. Removing a child queue affects container
assignment, but that is a good thing. 
We should probably add a comment to that effect so we don't forget this in the future.

bq. instead of using a List and sorting it everytime, we could use a Sorted Bag (MultiSet)
? 
One issue with using a sorted list is the sorting happens on addition/removal. FSQueues already
in the list also change affecting the "order". May be, we could remove and re-insert the queue
if anything changes, but that is a much bigger change and needs to be carefully evaluated
for performance.

> FairScheduler: CME on FSParentQueue#getQueueUserAclInfo
> -------------------------------------------------------
>
>                 Key: YARN-3762
>                 URL: https://issues.apache.org/jira/browse/YARN-3762
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler
>    Affects Versions: 2.7.0
>            Reporter: Karthik Kambatla
>            Assignee: Karthik Kambatla
>            Priority: Critical
>         Attachments: yarn-3762-1.patch, yarn-3762-1.patch
>
>
> In our testing, we ran into the following ConcurrentModificationException:
> {noformat}
> halxg.cloudera.com:8042, nodeRackName/rackvb07, nodeNumContainers0
> 15/05/22 13:02:22 INFO distributedshell.Client: Queue info, queueName=root.testyarnpool3,
queueCurrentCapacity=0.0, queueMaxCapacity=-1.0, queueApplicationCount=0, queueChildQueueCount=0
> 15/05/22 13:02:22 FATAL distributedshell.Client: Error running Client
> java.util.ConcurrentModificationException: java.util.ConcurrentModificationException
> 	at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
> 	at java.util.ArrayList$Itr.next(ArrayList.java:851)
> 	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSParentQueue.getQueueUserAclInfo(FSParentQueue.java:155)
> 	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.getQueueUserAclInfo(FairScheduler.java:1395)
> 	at org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getQueueUserAcls(ClientRMService.java:880)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message