Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of kengoodhope@gmail.com
 designates 74.125.83.176 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=nKvmU1SyUkZTXKP0a2PQS1wKnpr/n3xBuoHbIHcHDQRznitMmI+WTOU9HrtquZd22P
         2HhvgtM9SVWI/gJOXx+oAYu1f7ZaTwfqxi2L948Ji3pgtAQVoRhjPNZ3tFlUmpf+YU3H
         4gTRPVSnVHGegetlpFNl2rAG5slTb/0dgrHrY=
MIME-Version: 1.0
In-Reply-To: <AANLkTikmt1dEDZcaJn_o8qVt55VmCRE7tpGO9PvHJRMI@mail.gmail.com>
References: <AANLkTikmt1dEDZcaJn_o8qVt55VmCRE7tpGO9PvHJRMI@mail.gmail.com>
Date: Sun, 4 Jul 2010 12:45:05 -0700
Message-ID: <AANLkTildkNkkOZ45g8N5voO4g7SOk5uy10-8PZ6PdezN@mail.gmail.com>
Subject: Re: why my Reduce Class does not work?
From: Ken Goodhope <kengoodhope@gmail.com>
To: common-user@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1

You need @Override on your reduce method. Right now you are getting
the identity reduce method.

On 7/4/10, Vitaliy Semochkin <vitaliy.se@gmail.com> wrote:
> Hi,
>
> I rewritten WordCount sample to use new Hadoop API
>
> however my reduce task doesn't launch.
>
> the result file always looks like
> some_word 1
> some_word 1
> another_word 1
> another_word 1
>
> ...
>
> Here is the code:
>
> import java.io.IOException;
> import java.util.StringTokenizer;
>
> import org.apache.hadoop.fs.Path;
> import org.apache.hadoop.io.IntWritable;
> import org.apache.hadoop.io.LongWritable;
> import org.apache.hadoop.io.Text;
> import org.apache.hadoop.mapreduce.Job;
> import org.apache.hadoop.mapreduce.Mapper;
> import org.apache.hadoop.mapreduce.Reducer;
> import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
> import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
>
> public class WordCount {
>
> public static class WordCountMapper extends Mapper<LongWritable, Text, Text,
> IntWritable> {
>
> @Override
> protected void map(LongWritable key, Text value, Context context) throws
> IOException, InterruptedException {
> StringTokenizer st = new StringTokenizer(value.toString());
> while (st.hasMoreTokens()) {
> context.write(new Text(st.nextToken()), new IntWritable(1));
> }
> }
> }
>
> public static class WordCountReduce extends Reducer<Text, IntWritable, Text,
> IntWritable> {
>
> @SuppressWarnings("unchecked")
> public void reduce(Text key, Iterable<IntWritable> values, Reducer.Context
> context) throws IOException, InterruptedException {
> int sum = 0;
> for (IntWritable value : values) {
> sum += value.get();
> }
> context.write(key, new IntWritable(sum));
> }
> }
>
> public static void main(String[] args) throws IOException,
> InterruptedException, ClassNotFoundException {
> Job job = new Job();
> job.setJobName("WordCounter");
> job.setJarByClass(WordCount.class);
> job.setMapperClass(WordCountMapper.class);
> job.setReducerClass(WordCountReduce.class);
> job.setOutputKeyClass(Text.class);
> job.setOutputValueClass(IntWritable.class);
> FileInputFormat.setInputPaths(job, new Path(args[0]));
> FileOutputFormat.setOutputPath(job, new Path(args[1]));
> System.exit(job.waitForCompletion(true)? 0 :1);
> }
> }
>
> Looks like WordCountReduce was never launched but I don't see any Warnings
> or Errors in log  file.
>
> Any help is highly appreciated.
>
> Thanks in Advance,
> Vitaliy S
>