hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aseem Anand <aseem.ii...@gmail.com>
Subject QuickSort array out of bound exception during Secondary Sort
Date Tue, 05 Feb 2013 13:11:01 GMT
Hi,
I'm facing a problem with hadoop's secondary sort such that it is
displaying the following error message. The code I have used has been used
by me previously and had not given any issues for a large number of
intermediate output pairs. The number of mapper output values is around
175,000. Could someone please guide me on this .

13/02/05 18:34:27 WARN mapred.LocalJobRunner: job_local_0001
java.lang.ArrayIndexOutOfBoundsException: 1
at org.skyz.basic.KeyComparator.compare(KeyComparator.java:23)
at
org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:106)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.compare(MapTask.java:1111)
at org.apache.hadoop.util.QuickSort.fix(QuickSort.java:30)
at org.apache.hadoop.util.QuickSort.sortInternal(QuickSort.java:83)
at org.apache.hadoop.util.QuickSort.sort(QuickSort.java:59)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1399)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1298)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:699)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:766)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)

Hoping for a positive response.
Thanks & Regards,
Aseem Anand

Mime
View raw message