hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Randy Prager (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-1470) map/red jobs fail using G1 GC (Couldn't find heap)
Date Mon, 28 Jun 2010 21:18:50 GMT

    [ https://issues.apache.org/jira/browse/PIG-1470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12883302#action_12883302
] 

Randy Prager commented on PIG-1470:
-----------------------------------

thanks.  we started testing w/ G1 GC on our hadoop cluster to avoid (which it seems to do)
the exceptions

{noformat}
java.io.IOException: Task process exit with nonzero status of 134.
        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:418)
{noformat}

which occur randomly on 6u18,6u20 and the default GC.  We are going to try some other Java
version + GC combinations ... do you have any insight into a stable mix of Java versions and
GC?

> map/red jobs fail using G1 GC (Couldn't find heap)
> --------------------------------------------------
>
>                 Key: PIG-1470
>                 URL: https://issues.apache.org/jira/browse/PIG-1470
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.6.0
>         Environment: OS: 2.6.27.19-5-default #1 SMP 2009-02-28 04:40:21 +0100 x86_64
x86_64 x86_64 GNU/Linux
> Java: Java(TM) SE Runtime Environment (build 1.6.0_18-b07)
> Hadoop: 0.20.1
>            Reporter: Randy Prager
>
> Here is the hadoop map/red configuration (conf/mapred-site.xml) that fails
> {noformat}
>          <property>
>                 <name>mapred.child.java.opts</name>
>                 <value>-Xmx300m -XX:+DoEscapeAnalysis -XX:+UseCompressedOops -XX:+UnlockExperimentalVMOptions
-XX:+UseG1GC</value>
>         </property>
> {noformat}
> Here is the hadoop map/red configuration that succeeds
> {noformat}
>          <property>
>                 <name>mapred.child.java.opts</name>
>                 <value>-Xmx300m -XX:+DoEscapeAnalysis -XX:+UseCompressedOops</value>
>         </property>
> {noformat}
> Here is the exception from the pig script.
> {noformat}
> Backend error message
> ---------------------
> org.apache.pig.backend.executionengine.ExecException: ERROR 2081: Unable to set up the
load function.
>         at org.apache.pig.backend.executionengine.PigSlice.init(PigSlice.java:89)
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SliceWrapper.makeReader(SliceWrapper.java:144)
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getRecordReader(PigInputFormat.java:282)
>         at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:338)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
>         at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: java.lang.RuntimeException: could not instantiate 'PigStorage' with arguments
'[,]'
>         at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:519)
>         at org.apache.pig.backend.executionengine.PigSlice.init(PigSlice.java:85)
>         ... 5 more
> Caused by: java.lang.reflect.InvocationTargetException
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
>         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>         at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:487)
>         ... 6 more
> Caused by: java.lang.RuntimeException: Couldn't find heap
>         at org.apache.pig.impl.util.SpillableMemoryManager.<init>(SpillableMemoryManager.java:95)
>         at org.apache.pig.data.BagFactory.<init>(BagFactory.java:106)
>         at org.apache.pig.data.DefaultBagFactory.<init>(DefaultBagFactory.java:71)
>         at org.apache.pig.data.BagFactory.getInstance(BagFactory.java:76)
>         at org.apache.pig.builtin.Utf8StorageConverter.<init>(Utf8StorageConverter.java:49)
>         at org.apache.pig.builtin.PigStorage.<init>(PigStorage.java:69)
>         at org.apache.pig.builtin.PigStorage.<init>(PigStorage.java:79)
>         ... 11 more
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message