hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmitry" <dmitrytka...@hotmail.com>
Subject Re: Why no mock objects in Hadoop?
Date Mon, 24 Sep 2007 18:03:15 GMT
What actually better to use for Hadoop  - EasyMock or JMock ? Any comparison 
table  between them?

----- Original Message ----- 
From: "Nigel Daley" <ndaley@yahoo-inc.com>
To: <hadoop-user@lucene.apache.org>
Sent: Monday, September 24, 2007 12:56 PM
Subject: Re: Why no mock objects in Hadoop?

> On Sep 22, 2007, at 2:59 PM, kate rhodes wrote:
>> On 9/21/07, Nigel Daley <ndaley@yahoo-inc.com> wrote:
>>> All the junit tests in hadoop are system type tests (as you've
>>> noted).  But, since Mapper and Reducer method parameters are all
>>> interfaces, it should be easy to use something like EasyMock or jMock
>>> to mock the input parameters and test your classes.
>> You are correct... in part. There are two problems that I  discovered 
>> with this.
>> 1) JobConf does not implement an interface it extends Configuration
>> and the only thing Configuration implements is Iterable.
>> 2) and this is the scary one... There are NO mock object libraries OR
>> objects in Hadoop. OMFGWTFBBQ. Seriously guys. This means there's
>> *probably* not a single UNIT test in Hadoop, just lots of integration
>> and system tests. Not to diss the integration tests you guys have
>> written but they're NOT unit tests and that fact means that you're
>> just hurting yourselves when it comes time to debug because a bug in
>> one class has the potential to show up as a failure in any of the
>> other classes that use it in their tests, which can mislead you into
>> thinking that you've got a bug in a class that's perfectly fine.
> Welcome to Open Source :-)  Definitions on what unit tests are/aren't 
> w.r.t. to OOP have only recently started to converge around those 
> popularized by Martin Fowler, Kent Beck, et al. ('converge' is likely  too 
> strong a word).  It's possible that many developers aren't test  driven, 
> don't keep up on the test philosophy du jour, and/or don't  agree with 
> these definitions.  Most days I'm just grateful that our  agreed upon code 
> review guidelines include checks for tests:
> http://wiki.apache.org/lucene-hadoop/CodeReviewChecklist
>> Now, I've got no problems using EasyMock and / or jMock for testing my
>> personal apps I do it all the time, but it means that I can't add any
>> to Hadoop without convincing someone to add in one of those two jars
>> (actually in jMock's case it looks like a whole pile-o-jars).
> You can always submit it as part of a patch if the library is license 
> compatible:
> http://people.apache.org/~cliffs/3party.html
> Nige

View raw message