hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5513) ConcurrentModificationException in JobControl
Date Wed, 18 Sep 2013 14:29:52 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13770815#comment-13770815
] 

Jason Lowe commented on MAPREDUCE-5513:
---------------------------------------

Sample backtrace when this occurs in a Pig job:

{noformat}
org.apache.pig.backend.executionengine.ExecException: ERROR 2043: Unexpected error during
execution.
    at org.apache.pig.PigServer.launchPlan(PigServer.java:1286)
    at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1260)
    at org.apache.pig.PigServer.execute(PigServer.java:1250)
    at org.apache.pig.PigServer.executeBatch(PigServer.java:362)
    at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:132)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:193)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
    at org.apache.pig.Main.run(Main.java:435)
    at org.apache.pig.PigRunner.run(PigRunner.java:49)
    at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:283)
    at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:223)
    at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37)
    at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:76)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:494)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:400)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:335)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
Caused by: java.util.ConcurrentModificationException
    at java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:953)
    at java.util.LinkedList$ListItr.next(LinkedList.java:886)
    at org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.toList(JobControl.java:83)
    at org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.getSuccessfulJobList(JobControl.java:125)
    at org.apache.hadoop.mapred.jobcontrol.JobControl.getSuccessfulJobs(JobControl.java:75)
    at org.apache.pig.tools.pigstats.PigStatsUtil.accumulateStats(PigStatsUtil.java:244)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:304)
    at org.apache.pig.PigServer.launchPlan(PigServer.java:1275)
    ... 26 more
{noformat}
                
> ConcurrentModificationException in JobControl
> ---------------------------------------------
>
>                 Key: MAPREDUCE-5513
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5513
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 2.1.0-beta, 0.23.9
>            Reporter: Jason Lowe
>
> JobControl.toList is locking individual lists to iterate them, but those lists can be
modified elsewhere without holding the list lock.  The locking approaches are mismatched,
with toList holding the lock on the actual list object while other methods hold the JobControl
lock when modifying the lists.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message