drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timothy Farkas <tfar...@mapr.com>
Subject Re: Speeding Up Unit Tests
Date Tue, 29 Aug 2017 21:08:33 GMT
Thanks Paul,

I see now the parent pom already configures surefire to fork 2 processes to run tests. I tried
running multiple tests in parallel in each forked process, but there were errors with BaseTestQuery,
probably because it uses static variables. One additional speedup I found was that the maven
-T flag parallelizes the build and also seems to parallelize running the tests across sub
modules. Using that flag the test duration for contrib went from 18 minutes to 10 minutes.

I'll investigate making BaseTestQuery thread safe. Maybe we can get more speedups from being
able to run multiple tests in parallel in the same process as well.

Tim

________________________________
From: Paul Rogers <progers@mapr.com>
Sent: Monday, August 28, 2017 6:39:10 PM
To: dev@drill.apache.org
Subject: Re: Speeding Up Unit Tests

Tests run in parallel when run from the command line in Maven. We have had issues, especially
when some test changes global state. Can’t remember the details, however.

- Paul

> On Aug 28, 2017, at 6:04 PM, Timothy Farkas <tfarkas@mapr.com> wrote:
>
> Hi All,
>
> I will be working on DRILL-5730<https://issues.apache.org/jira/browse/DRILL-5730>
 and see some other areas for potential improvement with regard to testing:
>
>  1.  Add caching of maven artifacts to the Travis build. This should significantly speed
up compiling the code in travis.
>  2.  Running unit tests in parallel.
>  3.  Make the Travis build actually run unit tests (currently it does not).
>
> Points (1) and (3) are easy to do, but I was wondering if anyone has experience with
some of the potential pitfalls of running unit tests in parallel? Are there certain tests
which will have race conditions? and are there any foreseeable issues with running multiple
embedded drill bits in parallel?
>
> Thanks,
> Tim
>
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message