hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nigel Daley <nda...@yahoo-inc.com>
Subject Re: Why no mock objects in Hadoop?
Date Mon, 24 Sep 2007 17:56:01 GMT

On Sep 22, 2007, at 2:59 PM, kate rhodes wrote:

> On 9/21/07, Nigel Daley <ndaley@yahoo-inc.com> wrote:
>> All the junit tests in hadoop are system type tests (as you've
>> noted).  But, since Mapper and Reducer method parameters are all
>> interfaces, it should be easy to use something like EasyMock or jMock
>> to mock the input parameters and test your classes.
> You are correct... in part. There are two problems that I  
> discovered with this.
> 1) JobConf does not implement an interface it extends Configuration
> and the only thing Configuration implements is Iterable.
> 2) and this is the scary one... There are NO mock object libraries OR
> objects in Hadoop. OMFGWTFBBQ. Seriously guys. This means there's
> *probably* not a single UNIT test in Hadoop, just lots of integration
> and system tests. Not to diss the integration tests you guys have
> written but they're NOT unit tests and that fact means that you're
> just hurting yourselves when it comes time to debug because a bug in
> one class has the potential to show up as a failure in any of the
> other classes that use it in their tests, which can mislead you into
> thinking that you've got a bug in a class that's perfectly fine.

Welcome to Open Source :-)  Definitions on what unit tests are/aren't  
w.r.t. to OOP have only recently started to converge around those  
popularized by Martin Fowler, Kent Beck, et al. ('converge' is likely  
too strong a word).  It's possible that many developers aren't test  
driven, don't keep up on the test philosophy du jour, and/or don't  
agree with these definitions.  Most days I'm just grateful that our  
agreed upon code review guidelines include checks for tests:

> Now, I've got no problems using EasyMock and / or jMock for testing my
> personal apps I do it all the time, but it means that I can't add any
> to Hadoop without convincing someone to add in one of those two jars
> (actually in jMock's case it looks like a whole pile-o-jars).

You can always submit it as part of a patch if the library is license  


View raw message