hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-410) [testing] Speed up the test suite
Date Fri, 16 Oct 2009 16:47:31 GMT

    [ https://issues.apache.org/jira/browse/HBASE-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12766596#action_12766596
] 

stack commented on HBASE-410:
-----------------------------

I think we should close this issue.  Further speed up can only come from rewrite of individual
tests.

TestRegionRebalancing takes 2m22seconds.  Its going to go away in 0.21 when we break out the
balancing algorithm so its testable outside of a master/regionserver context.
TestZooKeeper takes 1m37seconds.  Could be rewritten but does stuff like shutting down regionservers.
 Can make an issue to rewrite or just wait until cluster becomes more live and then this test
time may go down significantly.
Long-running client package tests are doing a lot.  Will get faster when cluster becomes more
live.
MR could be redone to not write so much data but then again, its testing a bunch of stuff....
TestLogRolling takes 1m17s.  Its writing a bunch of data.
Thrift takes > 1m.

Thats it. 

We need more tests.  

> [testing] Speed up the test suite
> ---------------------------------
>
>                 Key: HBASE-410
>                 URL: https://issues.apache.org/jira/browse/HBASE-410
>             Project: Hadoop HBase
>          Issue Type: Test
>          Components: test
>    Affects Versions: 0.2.0
>            Reporter: Bryan Duxbury
>            Priority: Minor
>             Fix For: 0.21.0
>
>         Attachments: 410-v2.patch, 410-v3.patch, 410-v4.patch, 410.patch
>
>
> The test suite takes a long time to run, and a lot of the time spent running is really
wasted on repeated startup and shutdown, waiting for asynchronous events to occur, and production-tuned
timeouts to occur. Additionally, we use a MiniDFS instead of the local filesystem. 
> We should:
>  - Make whatever changes are needed to the local DFS so that it can run on Windows and
use that as the basis of all of our tests
>  - Minimize redoing expensive setup where possible by combining tests into groups or
suites that can share common setup
>  - Create a way of running all the parts (Master, Regionserver, Client) in a single thread
and explicitly advancing through necessary states so that we can reliably and quickly get
what we need tested accomplished
>  - Use smaller test datasets where it would make a difference (TestTableIndex and TestTableMapReduce,
I'm looking at you!)
> A faster test suite means faster turnaround on new patches, faster Hudson, and a shorter
patch queue. Not to mention less annoyance on the part of the developers.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message