hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nguyen Manh Tien" <tien.nguyenm...@gmail.com>
Subject Multi-threaded Reduce
Date Thu, 04 Oct 2007 01:50:15 GMT
I know in Hadoop we can implement multi-threaded, asynchronous mapping with
class MapRunnable. But this don't exist the  similar class to do
multi-threaded in reduce phrase. Could we do milti-thread in reduce phrase?.
Does the following code work?

public void reduce(WritableComparable key, Iterator values,
                     OutputCollector output, Reporter reporter) {
     new SomeThread(output).start(); // transfer OutputCollector to thread
}

public class SomeThread extend Thread {
   OutputCollector ouput;
   public SomeThread(OutputCollector output) {
     this.output = output;
   }
   public void run() {
     output.collect(key, value);
   }
}

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message