lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: (LUCENE-1844) Speed up junit tests
Date Thu, 26 Nov 2009 10:14:55 GMT
On Wed, Nov 25, 2009 at 8:29 PM, Erick Erickson <erickerickson@gmail.com> wrote:
> IMHO there are other reasons to upgrade to junit4 besides
> parallelization, there are some nice new capabilities. I
> suppose the analogous question is "why upgrade to
> Lucene 2.9?"

OK then I think we should do the upgrade, and separately think about
how to take advantage of parallelization... I applied the patch for
1084, and also carried it to the back-compat branch, and upgraded the
junit jar in the lib dir to 4.7, and all tests run fine.  I also
separately tested 2037, and it ran fine as well.

> Especially since it's not a matter of upgrading. Junit3 tests run
> just fine under junit4. I've tested after removing the
> junit3 jar from lib, no problem. It even seems to run
> slightly faster, which makes me wonder...
>
> So really, we have the best of both worlds. No work involved in
> using Junit4 with the current tests, but the ability to use the
> new features of Junit4. Although I'm sure there'll be
> *something* that bites us, I have great faith in Murphy.
>
> Kinda reminds me of the Lucene drop-in replacement policy
> <G>.......

Heh ;)

> But on the topic of parallelization: I'm not at all sure
> it's worth the effort. As far as I can tell, it really only gets significant
> gains when you have more cores to run with. It's not at all clear
> to me how much time we spend doing I/O in the tests... very little
> I suspect (although I confess I don't know for sure). And if we're
> CPU bound anyway, parallelization doesn't help. Anybody know
> for sure?
>
> And say we did all the work to parallelize all the tests. And say that
> instead of taking 25 minutes on my 3 year old MacBook Pro, we
> got it down to 10 minutes. Who cares? 10 minutes is still too long
> according to the eXtreme Programming (XP) folks, and I sympathize
> with their point of view. Even though I did  spend some time trying
> to trim some time.

If you have enough cores, you can get decent speedups.  I have a scary
python wrapper that runs the tests in parallel (runs 5 threads on an 8
core machine... any more threads seems not to help), by letting each
thread run a separate package, and sub-dividing the big (index,
search) packages w/ a temporary patch.  Still it takes ~6.5 minutes to
run the tests on my machine these days, which is still too long for
interactive testing...

> The XP approach to unit testing is to run it almost every time you
> change a line of code. OK, I'm exaggerating, but not by too
> much with the die-hard XP folks. Even at 10 minutes, we can't
> do that.
>
> So,  I think the value for Lucene/SOLR comes *not* from running the
> tests 15 times an hour. I think the real value comes from not
> letting errors hide for days/weeks/months/releases. So I'm quite willing
> to let the automated builds catch the unit test failures in unexpected
> places in those instances where I don't run all of the tests before
> a patch is committed. As long as we fix them as soon as they're
> found.

Kinda like what's the point of a cell phone having better battery life
when it's already plenty for my daily usage yet not enough for 2 days
usage, ie, I still have to charge it up, nightly.... it's quantized.
Slightly faster tests won't really make a fundamental change in our
development approach.

But, still, while this speedup won't make the tests fully interactive,
it's still an important improvement.  I run the unit tests an insane
number times, and bringing the time down even "just" by 2 minutes will
make a real difference.

> OK, I'm rambling. I'm off for Thanksgiving, and my daughter is
> at her in-laws until tomorrow (they're visiting from CA). So sue
> me <G>.

Happy Thanksgiving!!

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message