hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jesse Yates <jesse.k.ya...@gmail.com>
Subject Re: Running UnitTests before submitting a patch
Date Wed, 21 Sep 2011 22:59:01 GMT
Hi all,

Added https://issues.apache.org/jira/browse/HBASE-4454 to track just adding
the failsafe plugin to the regular build process. This is a stopgap till we
do the module split (which is going to be painful), to help speed up
people's builds.

-Jesse Yates

On Tue, Sep 20, 2011 at 7:22 AM, Doug Meil <doug.meil@explorysmedical.com>wrote:

>
> Hi there-
>
> https://issues.apache.org/jira/browse/HBASE-4448
>
> I attached a prototype of HBaseTestingUtilityFactory to this ticket.  I'd
> like to get some feedback on the general idea (emphasis on "general idea")
> and then if the consensus is that it makes sense I will continue and then
> we can do a more formal review on the reviews thing.
>
> Doug
>
>
>
>
> On 9/19/11 11:55 PM, "Stack" <stack@duboce.net> wrote:
>
> >On Mon, Sep 19, 2011 at 1:06 PM, Jesse Yates <jesse.k.yates@gmail.com>
> >wrote
> >>> ... will spent (at least on my laptop) about 10.7 seconds setting up
> >>>the
> >>> cluster, and 7.3 seconds tearing down.  Assuming that we aren't
> >>>running in
> >>> a separate JVM each test invocation, sharing the same instance of
> >>> HBaseTestingUtility would be an enormous benefit.
> >>>
> >
> >Seems like it would be worth investigating makeing this run
> >
> >
> >>> 1)  Re-use JVMs between test runs (if we are already doing this, then
> >>>no
> >>> action)
> >>>
> >>
> >> This is already built into maven when we run tests normally. This leads
> >>to
> >> some interesting situations for messing around with class loading/global
> >> variables; if you don't reset the environment, it can cause other
> >> (essentially non-broken) tests to fail.  Unless we are running things in
> >> forked mode .
> >>
> >> Forked mode could also cut down on the straight unit test time, but in
> >>my
> >> experience this has lead to difficulty in figuring out what crashed. For
> >> some reason maven doesn't realize which tests failed, just that they
> >>did, in
> >> its final output (though it still stdouts "FAILURE").
> >>
> >
> >Yeah, we run forked mode.
> >
> >          <configuration>
> >
> ><forkedProcessTimeoutInSeconds>900</forkedProcessTimeoutInSeconds>
> >            <argLine>-enableassertions -Xmx1400m</argLine>
> >            <redirectTestOutputToFile>true</redirectTestOutputToFile>
> >          </configuration>
> >
> >I think the thinking was fork it because so much is going on and yeah,
> >for sure there'll be cleanup afterward if all goes awry.
> >
> >Lohit suggested running tests in parallel.  I tried it but things
> >didn't seem to run faster in my playing around.  See HBASE-4207.
> >
> >
> >>> 2)  Re-use the same HBaseTestingUtility instance for all possible
> >>>tests.
> >>>
> >>> I think #2 isn't "that bad" if we had a factory method to get/return
> >>> HBaseTestingUtility instance and internally this factory could do a
> >>> ref-count  and auto-detect of non-activity, then it could shutdown the
> >>> cluster behind the scenes.
> >>>
> >>>
> >> +1 on this idea.
> >> We should also look into adding more/less region servers whenever people
> >> request more. We could just leave it up for the entire span of the
> >> integration tests and then bring it down afterwards (to make it easier
> >>on
> >> ourselves.
> >>
> >
> >We do some of this now.   TestFromClientSide and TestAdmin are an
> >aggregation of a bunch of tests that used to each spin up own
> >clusters.
> >
> >We should do more of this.  I think what happens is that a bunch of
> >our tests do one test only and to run the test, they spin up and down
> >a cluster.  Could put a few together -- those that make sense.
> >
> >St.Ack
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message