hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: Running UnitTests before submitting a patch
Date Tue, 20 Sep 2011 03:55:25 GMT
On Mon, Sep 19, 2011 at 1:06 PM, Jesse Yates <jesse.k.yates@gmail.com> wrote
>> ... will spent (at least on my laptop) about 10.7 seconds setting up the
>> cluster, and 7.3 seconds tearing down.  Assuming that we aren't running in
>> a separate JVM each test invocation, sharing the same instance of
>> HBaseTestingUtility would be an enormous benefit.

Seems like it would be worth investigating makeing this run

>> 1)  Re-use JVMs between test runs (if we are already doing this, then no
>> action)
> This is already built into maven when we run tests normally. This leads to
> some interesting situations for messing around with class loading/global
> variables; if you don't reset the environment, it can cause other
> (essentially non-broken) tests to fail.  Unless we are running things in
> forked mode .
> Forked mode could also cut down on the straight unit test time, but in my
> experience this has lead to difficulty in figuring out what crashed. For
> some reason maven doesn't realize which tests failed, just that they did, in
> its final output (though it still stdouts "FAILURE").

Yeah, we run forked mode.

            <argLine>-enableassertions -Xmx1400m</argLine>

I think the thinking was fork it because so much is going on and yeah,
for sure there'll be cleanup afterward if all goes awry.

Lohit suggested running tests in parallel.  I tried it but things
didn't seem to run faster in my playing around.  See HBASE-4207.

>> 2)  Re-use the same HBaseTestingUtility instance for all possible tests.
>> I think #2 isn't "that bad" if we had a factory method to get/return
>> HBaseTestingUtility instance and internally this factory could do a
>> ref-count  and auto-detect of non-activity, then it could shutdown the
>> cluster behind the scenes.
> +1 on this idea.
> We should also look into adding more/less region servers whenever people
> request more. We could just leave it up for the entire span of the
> integration tests and then bring it down afterwards (to make it easier on
> ourselves.

We do some of this now.   TestFromClientSide and TestAdmin are an
aggregation of a bunch of tests that used to each spin up own

We should do more of this.  I think what happens is that a bunch of
our tests do one test only and to run the test, they spin up and down
a cluster.  Could put a few together -- those that make sense.


View raw message