hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Zhang <zjf...@gmail.com>
Subject Bug of InputSampler
Date Fri, 08 Apr 2011 08:33:45 GMT
Hi all,

I found a probable bug of InputSampler in method writePartitionFile

    for(int i = 1; i < numPartitions; ++i) {
      int k = Math.round(stepSize * i);
      while (last >= k && comparator.compare(samples[last], samples[k]) ==
0) {
        ++k;
      }
      writer.append(samples[k], nullValue);
      last = k;
    }


In the line of writer.append(samples[k], nullValue), the k may be already
out of array index. maybe we should add one line to test whether it has been
out of array index as following
  for(int i = 1; i < numPartitions; ++i) {
      int k = Math.round(stepSize * i);
      while (last >= k && comparator.compare(samples[last], samples[k]) ==
0) {
        ++k;
      }
      if (k>=samples.length)
         break;

      writer.append(samples[k], nullValue);
      last = k;
    }


-- 
Best Regards

Jeff Zhang

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message