hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kumudu harshani <kumuduharsh...@gmail.com>
Subject Re: How to set 2mappers on 1 job
Date Sun, 23 Sep 2012 04:04:03 GMT
I am sorry.. I didn't get you.. shouldn't i handle that with jobconf code.

The confusion i have is, if i put like:
JobConf conf2 = new JobConf(WordCount.class);
Job job2 = new Job(conf2);
conf2.setOutputKeyClass(IntWritable.class);
conf2.setOutputValueClass(Text.class);

conf2.setMapperClass(Map1.class);
conf2.setReducerClass(Reduce1.class);

 ---it will execute Map1.class and then Reduce1.class.

so if i have Mapper1a.class and Mapper2a.class, how should i write the code
of job to execute both and then execute Reducer.class such that, Reducer
will take both mappers (1a, 1b) emit outputs...

thanks
kumudu

On Sun, Sep 23, 2012 at 9:23 AM, Bertrand Dechoux <dechouxb@gmail.com>wrote:

> You can use the map.input.file property to decide which logic should your
> mapper apply.
> Regards
> Bertrand
>
>
> On Sun, Sep 23, 2012 at 5:40 AM, kumudu harshani <kumuduharshani@gmail.com
> > wrote:
>
>> Hi...
>> Could someone help me with following scenario..
>>
>> I want implement a job which should get 2 mapper outputs and send them to
>> 1 reducer. Attached image show the flow I wanted....
>>
>>
>>
>>
>> Normal flow is like:
>>
>> JobConf conf2 = new JobConf(WordCount.class);
>> Job job2 = new Job(conf2);
>> conf2.setOutputKeyClass(IntWritable.class);
>> conf2.setOutputValueClass(Text.class);
>>
>> conf2.setMapperClass(Map1.class);
>> conf2.setReducerClass(Reduce1.class);
>>
>> --- where it takes 1 mapper and 1 reducer. What i want is to set 2
>> maps(mapper1a, mapper1b) and 1 reducer...
>> Is that possible, if so could someone please help..
>>
>> thanks
>> kumudu
>> --
>>
>> *Kumudu Samarappuli* | Creative Search Technologies, a Microsoft IEG
>> Partner | Software Engineer I | m: +94 719 258 242 |
>> www.microsoft.com/enterprisesearch
>>
>>
>
>
> --
> Bertrand Dechoux
>



-- 

*Kumudu Samarappuli* | Creative Search Technologies, a Microsoft IEG
Partner | Software Engineer I | m: +94 719 258 242 |
www.microsoft.com/enterprisesearch

Mime
View raw message