hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefan Groschupf ...@101tec.com>
Subject Re: test suite
Date Fri, 07 Mar 2008 02:10:55 GMT
Is there an hudson that build hbase?
Hudson should run the integration test suite with mini cluster and dfs  
where ant test for the user should only run local fs and local map  
reduce jobrunner.

On Mar 6, 2008, at 5:23 PM, Bryan Duxbury wrote:

> I also vote for the local file system. However, I understand that  
> there are some issues with local filesystem implementation on  
> Windows, and there are some bugs that will only manifest when run on  
> hdfs, mini or otherwise.
> I think the best middle ground is expecting an externally configured  
> dfs, local or otherwise, rather than spinning one up for every test.  
> This would require an extra step on the part of developers running  
> the test suite. Do you think that would make it notably more  
> difficult for new developers to get the test suite going?
> Whatever we end up doing regarding filesystem spin up, my testing  
> shows that save us maybe a minute over the entire suite. It's  
> certainly still a worthwhile task to pursue, but won't be a magic  
> bullet.
> -Bryan
> On Mar 6, 2008, at 5:11 PM, Stefan Groschupf wrote:
>> jmock2 is pretty useful, I agree.
>> Also we in pig I submitted a patch that uses the hadoop  
>> localjobrunner (jobtracker=local) - significant faster than the  
>> minicluster.
>> Also in case hbase uses hadoop configuration why not just using the  
>> local file system?
>> On Mar 6, 2008, at 4:45 PM, Bryan Duxbury wrote:
>>> I would add that we have an issue open for speeding up the test  
>>> suite, HBASE-410. Probably the most promising idea for speeding up  
>>> the test suite is converting some of the tests that don't need to  
>>> be integration tests into something more like true unit tests, or  
>>> specs for that matter. Mock objects used to test hyper-focused  
>>> bits of the system would be absolutely ideal, and has the  
>>> advantage of being quite fast.
>>> If you wanted to make some early contributions, anything in this  
>>> area would be a FANTASTIC place to put them.
>>> -Bryan
>>> On Mar 6, 2008, at 4:28 PM, stack wrote:
>>>> Stefan Groschupf wrote:
>>>>> Hi there,
>>>>> I checked out latest svn and did run the test.
>>>>> The test suite failed at TestTable, are there known problems  
>>>>> today?
>>>> Supposed to work.  Usually does.  New hadoop was checked in last  
>>>> night.  That might have been responsible.  Breaks everytime?  If  
>>>> so, open JIRA.
>>>>> Also the test suite took 24 min to run on my little poor box -  
>>>>> is there any way to speed this up?
>>>> Yeah.  Its a real prob.  Test suite starts up little mini dfs's  
>>>> and mini hbase clusters and sometimes mini mr clusters.  I've  
>>>> spent time on figuring how to do weird things like start the dfs  
>>>> cluster inside of an ant task but ran into classloading issues  
>>>> (joy).  Needs to be fixed.
>>>> (Our 'unit tests' are more like 'integration tests' -- one  
>>>> approache would be to call them so and move them out of the unit  
>>>> test suite).
>>>> St.Ack
>>>>> Thanks.
>>>>> Stefan
>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>> 101tec Inc.
>>>>> Menlo Park, California, USA
>>>>> http://www.101tec.com
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> 101tec Inc.
>> Menlo Park, California, USA
>> http://www.101tec.com

101tec Inc.
Menlo Park, California, USA

View raw message