hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5214) ConcurrentModificationException in FairScheduler.getTotalSlots
Date Wed, 11 Feb 2009 04:54:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12672509#action_12672509

Vinod K V commented on HADOOP-5214:

Here's the trace.
2009-02-08 16:27:57,496 ERROR org.apache.hadoop.mapred.FairScheduler: Failed to update fair
share calculations
        at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
        at java.util.HashMap$ValueIterator.next(HashMap.java:822)
        at org.apache.hadoop.mapred.FairScheduler.getTotalSlots(FairScheduler.java:703)
        at org.apache.hadoop.mapred.FairScheduler.updateFairShares(FairScheduler.java:622)
        at org.apache.hadoop.mapred.FairScheduler.update(FairScheduler.java:358)
        at org.apache.hadoop.mapred.FairScheduler$UpdateThread.run(FairScheduler.java:212)
2009-02-08 16:27:57,500 ERROR org.apache.hadoop.mapred.JobTracker: Tracker Expiry Thread got
exception: java.lang.NullPointerException

And the underlying problem is that getTotalSlots gets the list of trackers via TaskTrackerManager.taskTrackers()
and iterates through it to calculate the total number of map and reduce slots in the cluster.
The exception occurs when JT internally modifies the list of TaskTrackers for e.g while updating
the list of lost TaskTrackers.

This information is already available via ClusterStatus. Using it will fix the issue.

> ConcurrentModificationException in FairScheduler.getTotalSlots
> --------------------------------------------------------------
>                 Key: HADOOP-5214
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5214
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: contrib/fair-share
>            Reporter: Vinod K V

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message