Return-Path: Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: (qmail 71336 invoked from network); 26 Oct 2009 02:57:11 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 26 Oct 2009 02:57:11 -0000 Received: (qmail 30019 invoked by uid 500); 26 Oct 2009 02:57:11 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 29957 invoked by uid 500); 26 Oct 2009 02:57:11 -0000 Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-user@hadoop.apache.org Delivered-To: mailing list mapreduce-user@hadoop.apache.org Received: (qmail 29948 invoked by uid 99); 26 Oct 2009 02:57:11 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 26 Oct 2009 02:57:11 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of zjffdu@gmail.com designates 209.85.216.198 as permitted sender) Received: from [209.85.216.198] (HELO mail-px0-f198.google.com) (209.85.216.198) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 26 Oct 2009 02:57:03 +0000 Received: by pxi36 with SMTP id 36so2619386pxi.2 for ; Sun, 25 Oct 2009 19:56:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=YXgDnbgGBXr+C8bQwKsZHMPs9jpItiawiqxpiz+FfSE=; b=eT/wiT1q/fU8OdcexI4VDymW8aMXOKsPMpmNbp7ToxxdENXcmN+VfXMEShVo/rLQBR k/AFmKoINBxPxC6VakSDNkXskF4pvYgpct78btkaq7zzUcBP/HrJehrIzwHaHcCcNbUq j5NMSXRQIHfbyyZsO+uPKqn8dPwlZ0tRWmJgU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=CnIAydvj5NWCZXJOKtMP4eEWcA5fc2RPh1sheT1LLgKzjp/LjiNH3btXFH5Qetzwcp +9n3cMxShwmwjm9MX9UWCOYuYoE6ZGczTikf99YV5QqVs1BwVO007pHh4exjVOgl+la8 PMhNJ+EBZ6gdd1F3jAa+HHD7ICJjuidjaRUWg= MIME-Version: 1.0 Received: by 10.142.120.1 with SMTP id s1mr517165wfc.245.1256525801742; Sun, 25 Oct 2009 19:56:41 -0700 (PDT) In-Reply-To: References: Date: Mon, 26 Oct 2009 10:56:41 +0800 Message-ID: <8211a1320910251956i2846caf3x572a74232329b76f@mail.gmail.com> Subject: Re: Question regarding wordCount example From: Jeff Zhang To: mapreduce-user@hadoop.apache.org Content-Type: multipart/alternative; boundary=001636e0b9c610630a0476cdb636 X-Virus-Checked: Checked by ClamAV on apache.org --001636e0b9c610630a0476cdb636 Content-Type: text/plain; charset=UTF-8 Hi gao, You did not provider the type of key and value explicitly in your code, so you have to write your map method as public void map(Object key, Object value, OutputCollector output, Reporter reporter) throws IOException { // TODO Auto-generated method stub } But I suggest you write your Mapper class like this public class WordCountMapper extends MapReduceBase implements Mapper{ public void map(Long key, Text value, OutputCollector output, Reporter reporter) throws IOException { // TODO Auto-generated method stub } } using generic as possible as you can Jeff zhang On Mon, Oct 26, 2009 at 6:35 AM, felix gao wrote: > Hi all, I have some question regarding how to compile a simple hadoop > program. > > setup > Java 1.6 > Ubuntu 9.02 > Hadoop 0.19.2 > > > //below is the mapper class > import java.io.IOException; > import java.util.StringTokenizer; > > import org.apache.hadoop.io.IntWritable; > import org.apache.hadoop.io.LongWritable; > import org.apache.hadoop.io.Text; > import org.apache.hadoop.io.Writable; > import org.apache.hadoop.io.WritableComparable; > import org.apache.hadoop.mapred.MapReduceBase; > import org.apache.hadoop.mapred.Mapper; > import org.apache.hadoop.mapred.OutputCollector; > import org.apache.hadoop.mapred.Reporter; > > public class WordCountMapper extends MapReduceBase > implements Mapper{ > > private final IntWritable one = new IntWritable(1); > private Text word = new Text(); > > public void map(WritableComparable key, Writable value, > OutputCollector output, Reporter reporter) throws IOException { > > String line = value.toString(); > StringTokenizer itr = new StringTokenizer(line.toLowerCase()); > while(itr.hasMoreTokens()) { > word.set(itr.nextToken()); > output.collect(word, one); > } > } > > } > > > > here is the error message: > javac -classpath "/home/hadoop-user/hadoop-0.19/hadoop-0.19.2-core.jar" > *.java > WordCountMapper.java:14: WordCountMapper is not abstract and does not > override abstract method > map(java.lang.Object,java.lang.Object,org.apache.hadoop.mapred.OutputCollector,org.apache.hadoop.mapred.Reporter) > in org.apache.hadoop.mapred.Mapper > public class WordCountMapper extends MapReduceBase > ^ > Note: Some input files use unchecked or unsafe operations. > Note: Recompile with -Xlint:unchecked for details. > 1 error > > Can someone tell me what is going on here. > > Thanks, > > Felix > > > --001636e0b9c610630a0476cdb636 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi gao,

You did not provider the type of key and value explicitly in= your code, so you have to write your map method as

public void map= (Object key, Object value, OutputCollector output,
=C2=A0=C2=A0=C2=A0 = =C2=A0=C2=A0=C2=A0 Reporter reporter) throws IOException {
=C2=A0=C2=A0=C2=A0 // TODO Auto-generated method stub
=C2=A0=C2=A0=C2=A0=
}
=C2=A0
But I suggest you write your Mapper class like this
public class WordCountMapper extends MapReduceBase
=C2=A0=C2=A0=C2= =A0 implements Mapper<Long,Text,Text,Long>{
=C2=A0=C2=A0=C2=A0=C2=A0 public void map(Long key, Text value, OutputCollec= tor<Text, Long> output,
=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 Repo= rter reporter) throws IOException {
=C2=A0=C2=A0=C2=A0 // TODO Auto-gene= rated method stub
=C2=A0=C2=A0=C2=A0
=C2=A0=C2=A0=C2=A0=C2=A0 }
}=

using generic as possible as you can



Jeff zhang


On Mon, Oct 26= , 2009 at 6:35 AM, felix gao <gre1600@gmail.com> wrote:
Hi all, I have some question regarding how to compile a simple hadoop progr= am.

setup
Java 1.6
Ubuntu 9.02
H= adoop 0.19.2


//below is the mapper class
import java.io.IOExc= eption;
import java.util.StringTokenizer;

import org.apache.hadoop.io.IntWri= table;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.io.Writable;<= br>import org.apache.hadoop.io.WritableComparable;
import org.apache.had= oop.mapred.MapReduceBase;
import org.apache.hadoop.mapred.Mapper;
import org.apache.hadoop.mapred.OutputCollector;
import org.apache.hadoo= p.mapred.Reporter;

public class WordCountMapper extends MapReduceBas= e
=C2=A0=C2=A0=C2=A0 implements Mapper{

=C2=A0 private final IntW= ritable one =3D new IntWritable(1);
=C2=A0 private Text word =3D new Text();

=C2=A0 public void map(Writ= ableComparable key, Writable value,
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Outpu= tCollector output, Reporter reporter) throws IOException {

=C2=A0=C2= =A0=C2=A0 String line =3D value.toString();
=C2=A0=C2=A0=C2=A0 StringTok= enizer itr =3D new StringTokenizer(line.toLowerCase());
=C2=A0=C2=A0=C2=A0 while(itr.hasMoreTokens()) {
=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 word.set(itr.nextToken());
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 output.= collect(word, one);
=C2=A0=C2=A0=C2=A0 }
=C2=A0 }

}


here is the error message:
javac -classpath "/home/hadoop-user= /hadoop-0.19/hadoop-0.19.2-core.jar" *.java
WordCountMapper.java:14: WordCountMapper is not abstract and does not overr= ide abstract method map(java.lang.Object,java.lang.Object,org.apache.hadoop= .mapred.OutputCollector,org.apache.hadoop.mapred.Reporter) in org.apache.ha= doop.mapred.Mapper
public class WordCountMapper extends MapReduceBase
=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 ^
Note: Some input files use unchecked or unsafe operati= ons.
Note: Recompile with -Xlint:unchecked for details.
1 error
Can someone tell me what is going on here.

Thanks,

Felix



--001636e0b9c610630a0476cdb636--