hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benjamin Reed (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-31) concurrent modification error
Date Wed, 28 Nov 2007 19:36:43 GMT

    [ https://issues.apache.org/jira/browse/PIG-31?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12546362
] 

Benjamin Reed commented on PIG-31:
----------------------------------

I think if one thread calls clear() on the DataBag the other with the iterator should get
a CoModificationException just like if one thread does an add.

> concurrent modification error
> -----------------------------
>
>                 Key: PIG-31
>                 URL: https://issues.apache.org/jira/browse/PIG-31
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>            Reporter: Olga Natkovich
>            Assignee: Olga Natkovich
>            Priority: Critical
>         Attachments: concurrentmod.patch
>
>
> Many users encountered this problem. The stack looks like:
> Error:java.lang.RuntimeException
> 	at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
> 	at java.util.AbstractList$Itr.next(AbstractList.java:343)
> 	at com.yahoo.pig.builtin.SUM.sum(SUM.java:58)
> 	at com.yahoo.pig.builtin.SUM.exec(SUM.java:25)
> 	at com.yahoo.pig.builtin.SUM.exec(SUM.java:21)
> 	at com.yahoo.pig.impl.eval.FuncEvalSpec$1.add(FuncEvalSpec.java:101)
> 	at com.yahoo.pig.impl.eval.GenerateSpec$CrossProductItem.(GenerateSpec.java:140)
> 	at com.yahoo.pig.impl.eval.GenerateSpec$1.add(GenerateSpec.java:52)
> 	at com.yahoo.pig.impl.eval.GenerateSpec$CrossProductItem.add(GenerateSpec.java:217)
> 	at com.yahoo.pig.impl.eval.collector.UnflattenCollector.add(UnflattenCollector.java:42)
> 	at com.yahoo.pig.impl.eval.collector.DataCollector.addToSuccessor(DataCollector.java:79)
> 	at com.yahoo.pig.impl.eval.SimpleEvalSpec$1.add(SimpleEvalSpec.java:21)
> 	at com.yahoo.pig.impl.eval.GenerateSpec$CrossProductItem.exec(GenerateSpec.java:248)
> 	at com.yahoo.pig.impl.eval.GenerateSpec$1.add(GenerateSpec.java:61)
> 	at com.yahoo.pig.impl.mapreduceExec.PigMapReduce.reduce(PigMapReduce.java:144)
> 	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:355)
> 	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1707)
> We traced the issue to the same bag being shared between multiple operations and clearing
it in one place caused problem in another. Will attach Ben's fix shortly. All unit and end-to-end
tests passed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message