hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Rosenstrauch <dar...@darose.net>
Subject Re: mrunit question
Date Thu, 12 Aug 2010 02:24:00 GMT
On 08/11/2010 08:14 PM, Aaron Kimball wrote:
> David,
>
> Since you are directly instantiating the Mapper and Reducer (not using
> ReflectionUtils), you are free to call setConf() yourself before you run the
> test.

Sort of.  What would wind up happening is that setConf would get called 
twice:  once by the mrunit framework with the empty config (for which 
I'd need to add special coding in to my reducer object to make it ignore 
it) and then a second time when I explicitly call it myself in my JUnit 
test.  It's not really ideal for me to have to add special code into my 
reducer to avoid this issue.


> If you're using the new API, there already is setConfiguration() and
> withConfiguration(), which will set the Configuration being passed into
> Context sent to your Mapper/Reducer's run() method.
> See https://issues.apache.org/jira/browse/mapreduce-1569 where this was
> introduced.
>
>
> - Aaron

Yes, just looked at that, and it looks like it exactly solves this 
issue.  Unfortunately, though, it says that fix is effective in v0.21. 
As we're running the stable CDH version (CDH2, which uses Hadoop v0.20) 
I can't take advantage of that fix yet.

Going to wind up forgoing unit tests on this M/R code for right now I think.

Thanks,

DR

Mime
View raw message