hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arsen Zahray <menk...@gmail.com>
Subject Re: Type mismatch in key from map when replacing Mapper with MultithreadMapper
Date Wed, 28 Sep 2011 06:55:53 GMT
Hey! Thank you for replying!

Please, confirm that I understand you correctly:
1. Use a class, which extends mapper
class MyMapper extends Mapper<LongWritable, Text, IntWritable, MyPage> {

    public void map(LongWritable key, Text value, Context context) throws
IOException, InterruptedException {
         //implement all logic here
        }
    }
2. Leave MultithreadMapper class empty:
public class MyMultithreadMapper extends MultithreadedMapper<LongWritable,
Text, IntWritable, MyPage> {

}
3. In main set
MultithreadedMapper.setMapperClass(job, MyMapper.class);

Should I set
job.setMapperClass(MyMapper.class);
or
job.setMapperClass(MyMultithreadMapper.class);
?

Arsen

On Wed, Sep 28, 2011 at 9:38 AM, Kamesh <kamesh.b@imaginea.com> wrote:

> **
> On Wednesday 28 September 2011 11:33 AM, Arsen Zahray wrote:
>
> MultithreadMapper extends MultithreadedMapper<LongWritable, Text,
> IntWritable, MyPage> {
>
>     ConcurrentLinkedQueue<MyScraper>    scrapers    = new
> ConcurrentLinkedQueue<MyScraper>();
>
>     public static final int                nThreads    = 5;
>
>     public MyMultithreadMapper() {
>         for (int i = 0; i < nT
>
> implement the map logic in a separate Mapper class and in the main method
> set the following property
> MultithreadedMapper.setMapperClass(job, MyMapper.class);
>
> --
> *Thanks&Regards,*
> *Bh.V.S.Kamesh*
>

Mime
View raw message