hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kamesh <kames...@imaginea.com>
Subject Re: Type mismatch in key from map when replacing Mapper with MultithreadMapper
Date Wed, 28 Sep 2011 07:01:43 GMT
On Wednesday 28 September 2011 12:25 PM, Arsen Zahray wrote:
> Hey! Thank you for replying!
>
> Please, confirm that I understand you correctly:
> 1. Use a class, which extends mapper
> class MyMapper extends Mapper<LongWritable, Text, IntWritable, MyPage> {
>
>     public void map(LongWritable key, Text value, Context context) 
> throws IOException, InterruptedException {
>          //implement all logic here
>         }
>     }
> 2. Leave MultithreadMapper class empty:
> public class MyMultithreadMapper extends 
> MultithreadedMapper<LongWritable, Text, IntWritable, MyPage> {
>
> }
> 3. In main set
> MultithreadedMapper.setMapperClass(job, MyMapper.class);
>
> Should I set
> job.setMapperClass(MyMapper.class);
> or
> job.setMapperClass(MyMultithreadMapper.class);
> ?
>
> Arsen
>
> On Wed, Sep 28, 2011 at 9:38 AM, Kamesh <kamesh.b@imaginea.com 
> <mailto:kamesh.b@imaginea.com>> wrote:
>
>     On Wednesday 28 September 2011 11:33 AM, Arsen Zahray wrote:
>>     MultithreadMapper extends MultithreadedMapper<LongWritable, Text,
>>     IntWritable, MyPage> {
>>
>>         ConcurrentLinkedQueue<MyScraper>    scrapers    = new
>>     ConcurrentLinkedQueue<MyScraper>();
>>
>>         public static final int                nThreads    = 5;
>>
>>         public MyMultithreadMapper() {
>>             for (int i = 0; i < nT
>     implement the map logic in a separate Mapper class and in the main
>     method set the following property
>     MultithreadedMapper.setMapperClass(job, MyMapper.class);
>
>     -- 
>     /Thanks&Regards,/
>     /Bh.V.S.Kamesh/
>
>
set the following property
job.setMapperClass(MyMultithreadMapper.class);

-- 
/Thanks&Regards,/
/Bh.V.S.Kamesh/

Mime
View raw message