hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Duxbury <br...@rapleaf.com>
Subject Re: test suite
Date Fri, 07 Mar 2008 01:23:52 GMT
I also vote for the local file system. However, I understand that  
there are some issues with local filesystem implementation on  
Windows, and there are some bugs that will only manifest when run on  
hdfs, mini or otherwise.

I think the best middle ground is expecting an externally configured  
dfs, local or otherwise, rather than spinning one up for every test.  
This would require an extra step on the part of developers running  
the test suite. Do you think that would make it notably more  
difficult for new developers to get the test suite going?

Whatever we end up doing regarding filesystem spin up, my testing  
shows that save us maybe a minute over the entire suite. It's  
certainly still a worthwhile task to pursue, but won't be a magic  
bullet.

-Bryan

On Mar 6, 2008, at 5:11 PM, Stefan Groschupf wrote:

> jmock2 is pretty useful, I agree.
> Also we in pig I submitted a patch that uses the hadoop  
> localjobrunner (jobtracker=local) - significant faster than the  
> minicluster.
> Also in case hbase uses hadoop configuration why not just using the  
> local file system?
>
> On Mar 6, 2008, at 4:45 PM, Bryan Duxbury wrote:
>
>> I would add that we have an issue open for speeding up the test  
>> suite, HBASE-410. Probably the most promising idea for speeding up  
>> the test suite is converting some of the tests that don't need to  
>> be integration tests into something more like true unit tests, or  
>> specs for that matter. Mock objects used to test hyper-focused  
>> bits of the system would be absolutely ideal, and has the  
>> advantage of being quite fast.
>>
>> If you wanted to make some early contributions, anything in this  
>> area would be a FANTASTIC place to put them.
>>
>> -Bryan
>>
>> On Mar 6, 2008, at 4:28 PM, stack wrote:
>>
>>> Stefan Groschupf wrote:
>>>> Hi there,
>>>> I checked out latest svn and did run the test.
>>>>
>>>> The test suite failed at TestTable, are there known problems today?
>>>
>>> Supposed to work.  Usually does.  New hadoop was checked in last  
>>> night.  That might have been responsible.  Breaks everytime?  If  
>>> so, open JIRA.
>>>
>>>> Also the test suite took 24 min to run on my little poor box -  
>>>> is there any way to speed this up?
>>>
>>> Yeah.  Its a real prob.  Test suite starts up little mini dfs's  
>>> and mini hbase clusters and sometimes mini mr clusters.  I've  
>>> spent time on figuring how to do weird things like start the dfs  
>>> cluster inside of an ant task but ran into classloading issues  
>>> (joy).  Needs to be fixed.
>>>
>>> (Our 'unit tests' are more like 'integration tests' -- one  
>>> approache would be to call them so and move them out of the unit  
>>> test suite).
>>>
>>> St.Ack
>>>
>>>> Thanks.
>>>> Stefan
>>>>
>>>>
>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>> 101tec Inc.
>>>> Menlo Park, California, USA
>>>> http://www.101tec.com
>>>>
>>>>
>>>
>>
>>
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 101tec Inc.
> Menlo Park, California, USA
> http://www.101tec.com
>
>


Mime
View raw message