hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brahma Reddy Battula <brahmareddy.batt...@huawei.com>
Subject RE: ArrayIndexOutOfBoundsException occurs from MapTask
Date Thu, 12 Sep 2013 01:59:34 GMT
I had also faced same issue, after chnaging "mapreduce.task.io.sort.mb" from 2047 to 512, then
jobs are executed success

It might help in your case also..


From: sam liu [samliuhadoop@gmail.com]
Sent: Thursday, September 12, 2013 9:37 AM
To: user@hadoop.apache.org
Subject: ArrayIndexOutOfBoundsException occurs from MapTask


We are using Hadoop-1.1.1. When run a big workload mapreduce application, it threw a ArrayIndexOutOfBoundsException
as below. Seems the issue was threw in line 'System.arraycopy(b, off, kvbuffer, bufindex,
gaplen);' in MapTask. I am not familiar with the code of MapTask.java, and want to know which
situations will cause this kind of exception?

Caused by: java.lang.ArrayIndexOutOfBoundsException
        at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1270)
        at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1193)
        at java.io.DataOutputStream.write(DataOutputStream.java:82)
        at java.io.DataOutputStream.writeInt(DataOutputStream.java:191)


View raw message