hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5214) ConcurrentModificationException in FairScheduler.getTotalSlots
Date Wed, 11 Feb 2009 04:54:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12672509#action_12672509
] 

Vinod K V commented on HADOOP-5214:
-----------------------------------


Here's the trace.
{code}
2009-02-08 16:27:57,496 ERROR org.apache.hadoop.mapred.FairScheduler: Failed to update fair
share calculations
java.util.ConcurrentModificationException
        at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
        at java.util.HashMap$ValueIterator.next(HashMap.java:822)
        at org.apache.hadoop.mapred.FairScheduler.getTotalSlots(FairScheduler.java:703)
        at org.apache.hadoop.mapred.FairScheduler.updateFairShares(FairScheduler.java:622)
        at org.apache.hadoop.mapred.FairScheduler.update(FairScheduler.java:358)
        at org.apache.hadoop.mapred.FairScheduler$UpdateThread.run(FairScheduler.java:212)
2009-02-08 16:27:57,500 ERROR org.apache.hadoop.mapred.JobTracker: Tracker Expiry Thread got
exception: java.lang.NullPointerException
{code}

And the underlying problem is that getTotalSlots gets the list of trackers via TaskTrackerManager.taskTrackers()
and iterates through it to calculate the total number of map and reduce slots in the cluster.
The exception occurs when JT internally modifies the list of TaskTrackers for e.g while updating
the list of lost TaskTrackers.

This information is already available via ClusterStatus. Using it will fix the issue.

> ConcurrentModificationException in FairScheduler.getTotalSlots
> --------------------------------------------------------------
>
>                 Key: HADOOP-5214
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5214
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: contrib/fair-share
>            Reporter: Vinod K V
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message