hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-5031) Maps hitting IndexOutOfBoundsException for higher values of mapreduce.task.io.sort.mb
Date Tue, 26 Feb 2013 16:26:13 GMT
Karthik Kambatla created MAPREDUCE-5031:
-------------------------------------------

             Summary: Maps hitting IndexOutOfBoundsException for higher values of mapreduce.task.io.sort.mb
                 Key: MAPREDUCE-5031
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5031
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: mrv2
    Affects Versions: 0.23.5, 2.0.3-alpha
            Reporter: Karthik Kambatla
            Assignee: Karthik Kambatla


While trying to reproduce MAPREDUCE-5028 on trunk, ran into what seems to be a different issue.
To reproduce:

Psuedo-dist mode: mapreduce.{map,reduce}.memory.mb=2048, mapreduce.{map,reduce}.java.opts=-Xmx2048m,
mapreduce.task.io.sort.mb=1280

The map tasks fail with the following error: 
{noformat}
Error: java.lang.IndexOutOfBoundsException at java.nio.Buffer.checkIndex(Buffer.java:512)
at java.nio.ByteBufferAsIntBufferL.put(ByteBufferAsIntBufferL.java:113) at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1141)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:686) at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
at org.apache.hadoop.examples.WordCount$TokenizerMapper.map(WordCount.java:47) at org.apache.hadoop.examples.WordCount$TokenizerMapper.map(WordCount.java:36)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:757)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:339) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1488) at
org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
{noformat}


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message