hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ramkrishna S Vasudevan <ramakrish...@huawei.com>
Subject FW: Speeding up tests
Date Tue, 04 Oct 2011 10:43:52 GMT
I think the problem that i mentioned could be due to like i was using maven
In maven 2.9 this problem will not occur.  If a testcase gets killed it will
continue with other testcases.


-----Original Message-----
From: Ramkrishna S Vasudevan [mailto:ramakrishnas@huawei.com] 
Sent: Tuesday, October 04, 2011 12:18 PM
To: dev@hbase.apache.org
Subject: RE: Speeding up tests

Hi Jesse
Thanks for the write up.
I am using the script in HBASE-4480 widely.  I have a problem
Sometimes some test cases gets killed by the maven as it took a long time
and those testcases don't have timeout property in them.

Now if such testcase dont get completed(hanging happens) then maven kills
entirely and we are not able to proceed with other testcases.
Could you help me in this?

And do you have a list of flaky testcases?  I have a list prepared may be
you can add on to it.


-----Original Message-----
From: Doug Meil [mailto:doug.meil@explorysmedical.com] 
Sent: Tuesday, October 04, 2011 6:04 AM
To: dev@hbase.apache.org
Subject: Re: Speeding up tests

Thanks Jesse.  Great write up!

On 10/3/11 4:55 PM, "Jesse Yates" <jesse.k.yates@gmail.com> wrote:

>Hey everyone,
>There has been a bunch of work recently on speeding up the testing to make
>it easier for developers to iterate quickly on new features fixes. Part of
>the problem is that the test suite takes anywhere from 1-2 hrs to run and
>have some apparently non-deterministic hanging of tests.
>TL;DR To speed it all up, the attack plan would be:
>(1) Move long running tests to be integration tests,
>(2) Use a build server for patches so people only run unit tests locally,
>(3) We add unit tests when integration tests are breaking a lot but unit
>tests pass,
>(4) Go from forked to single jvm unit tests,
>(5) Add in surefire parallelization
>(6) Entertain using HBaseTestingUtilFactory
>I recently chatted with Stack and Doug about ways around this. Here is
>we came up with:
>(1) Break up long running tests from medium and short (2 mins max) tests
>move the former to be IntegrationTests.  This was based on Todd's
>in HBASE-4438. Either naming them integration tests or long running
>functional tests, they would become part of the 'mvn verify' rather than
>'mvn test' suite of tests. Starting point would be to  use Doug's
>spreadsheet from HBASE-4448 and when its done, the script from HBASE-4480.
>Right now, that means a LOT of tests are going to shift, but it means when
>developers run 'mvn test' the amount of time spent running unit tests will
>be cut down dramatically (hopefully towards the sub 10 -15 mins range)
>There is an implicit problem here: if the soon to be integration tests
>capture functionality that is not covered by unit tests, then people may
>incorrectly think that they are not breaking things. Therefore, we would
>(2) and (3):
>(2) Add a patch continuous integration server that goes and actually
>and tests patches as they come in. This would run 'mvn verify' and ensure
>that the patch actually isn't breaking high level/complex functionality.
>would be a requirement before patches are committed that they pass this
>(3) If we find that the unit tests aren't covering a certain level of
>functionality that is constantly breaking on the build server, we add more
>unit tests of the breaking functionality to ensure the unit tests are more
>complete and provide more assurances to developers when running them.
>This would be an ongoing process of comparing the integration tests vs.
>unit tests.
>(4) Once we have a true unit test suite, we should be able to go from
>'forked' jvm mode back to a single jvm for running tests. Unit tests
>not do crazy fault injection, full failure scenarios, so they should be
>to cleanup after themselves. This means we are going to get some speedup
>from not spinning up a new jvm for each test.
>(5) Once we are running in non-forked mode, we can try turning on
>parallelized test execution in surefire, which does parallel runs in a
>single jvm.
>(6) Once things all run in a single jvm, using the HBaseTestUtilFactory
>(HBASE-4448) make sense reuse the mini clusters across tests.
>What does everyone think of this approach?
>-Jesse Yates

View raw message