hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1372) ConcurrentModificationException in JobInProgress
Date Fri, 15 Jan 2010 15:54:54 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800728#action_12800728
] 

Amar Kamat commented on MAPREDUCE-1372:
---------------------------------------

JobTracker.addHostToNodeMapping() doesnt need any synchronization because the first call it
makes is to NetworkTopology.getNode() which is synchonized internally.  JobTracker.hostnameToNodeMap
(which gets modified in JobTracker.addHostToNodeMapping()) is synchronized collection and
hence it doesnt need any explicit synchronization. Hence I think the [patch|https://issues.apache.org/jira/secure/attachment/12430341/M1372-1.patch]
Chris uploaded should suffice.

> ConcurrentModificationException in JobInProgress
> ------------------------------------------------
>
>                 Key: MAPREDUCE-1372
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1372
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker
>    Affects Versions: 0.20.1
>            Reporter: Amareshwari Sriramadasu
>            Priority: Blocker
>             Fix For: 0.21.0
>
>         Attachments: M1372-0.patch, M1372-1.patch, M1372-2.patch
>
>
> We have seen the following  ConcurrentModificationException in one of our clusters
> {noformat}
> java.io.IOException: java.util.ConcurrentModificationException
>         at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
>         at java.util.HashMap$KeyIterator.next(HashMap.java:828)
>         at org.apache.hadoop.mapred.JobInProgress.findNewMapTask(JobInProgress.java:2018)
>         at org.apache.hadoop.mapred.JobInProgress.obtainNewMapTask(JobInProgress.java:1077)
>         at org.apache.hadoop.mapred.CapacityTaskScheduler$MapSchedulingMgr.obtainNewTask(CapacityTaskScheduler.java:796)
>         at org.apache.hadoop.mapred.CapacityTaskScheduler$TaskSchedulingMgr.getTaskFromQueue(CapacityTaskScheduler.java:589)
>         at org.apache.hadoop.mapred.CapacityTaskScheduler$TaskSchedulingMgr.assignTasks(CapacityTaskScheduler.java:677)
>         at org.apache.hadoop.mapred.CapacityTaskScheduler$TaskSchedulingMgr.access$500(CapacityTaskScheduler.java:348)
>         at org.apache.hadoop.mapred.CapacityTaskScheduler.addMapTask(CapacityTaskScheduler.java:1397)
>         at org.apache.hadoop.mapred.CapacityTaskScheduler.assignTasks(CapacityTaskScheduler.java:1349)
>         at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2976)
>         at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message