hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@yahoo-inc.com>
Subject Re: Can I wire a mapper and reducer class?
Date Thu, 12 Nov 2009 08:14:26 GMT
You can't do that, at least not yet: https://issues.apache.org/jira/browse/MAPREDUCE-1183

Arun

On Nov 11, 2009, at 10:28 PM, Java Developer wrote:

> I want to know if I can "wire" mapper and reducer classes. By "wire"  
> I mean to use inversion of control (or dependency injection). For  
> example, I see in a lot of books and on the Map/Reduce tutorial page  
> (http://hadoop.apache.org/common/docs/r0.20.0/mapred_tutorial.html),  
> something like this:
>
> conf.setMapperClass(MyMapper.class);
> conf.setReducerClass(MyReducer.class);
>
> However, I want do something like this:
>
> MyMapper mapper = new MyMapper();
> mapper.setSomeProperty("something");
>
> MyReducer reducer = new MyReducer();
> reducer.setSomeProperty("somethingElse");
>
> JobConf conf = new JobConf();
> conf.setMapper(mapper);
> conf.setReducer(reducer);
>
> Is this possible? I'm still learning MapReduce so I don't know if  
> this is possible. But all examples that I have come across follow  
> the pattern on the tutorial page. There is a work around that I've  
> been forced to. In my mapper and reducer class, I can grab some  
> properties file and then fill in the values that I need (for example  
> for JDBC connection). I see the problem that when I run this on a  
> distributed environment, I'll need the properties files there as  
> well. Does the mapper and reducer classes have a reference back to  
> the JobConf object? If so, I suppose I can get the properties there.
>
> Thanks.


Mime
View raw message