commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Phil Steitz <phil.ste...@gmail.com>
Subject Re: [MATH] speeding up unit tests
Date Thu, 18 Jul 2013 21:33:43 GMT
On 7/18/13 2:15 PM, Gilles wrote:
> On Thu, 18 Jul 2013 20:35:53 +0100, sebb wrote:
>> On 18 July 2013 20:06, Gilles <gilles@harfang.homelinux.org> wrote:
>>> On Thu, 18 Jul 2013 16:19:10 +0100, sebb wrote:
>>>>
>>>> Some MATH unit tests take a long time to run.
>>>>
>>>> One example is CMAESOptimizerTest, which is run twice - once
>>>> for the
>>>> deprecated code and once for the new code.
>>>>
>>>> Would it make sense to exclude the long-running tests of
>>>> deprecated
>>>> classes by default?
>>>>
>>>> Or maybe all the tests of deprecated classes should be skipped
>>>> by default?
>>>>
>>>> In the case where entire packages have been deprecated, that
>>>> would be
>>>> fairly easy.
>>>
>>>
>>> By @Ignore'ing the tests for the deprecated classes, we'd run the
>>> risk that a release contains failing code.
>>
>> That is not what I am suggesting.
>
> I didn't get what you were suggesting.
>
>>> Sometimes deprecated code must be touched, e.g. to redirect the
>>> internals toward new implementations (delegation). Bugs could be
>>> introduced unknowingly.
>>> Also the conversion of unit tests is not always trivial, and it is
>>> very useful to see that converted units tests run in the same way
>>> as those of the deprecated code.
>>>
>>> Isn't it possible to tell Junit to not run some tests (i.e. the
>>> opposite of "-Dtest=SomeTest")?
>>
>> Possibly, but there are too many to make it convenient, and who is
>> going to remember which of the CMAES tests to disable and which
>> needs
>> to be run?
>> They both have the same name; they are in different packages.
>>
>> I'm suggesting using a profile which is disabled by default.
>> In the same way that we don't run performance tests every time, 
>> maybe
>> it's not necessary to run deprecated class tests every time.
>
> So, are you proposing to rename e.g.
>   CMAESOptimizerTest
> to
>   CMAESOptimizerTestDeprecated
> ?
>
> Then I would need to explicitly require the tests to be run. If
> possible,
> I'd prefer the other way around run everything except when told
> otherwise.[1]
> Could this be possible with a rename like
>   CMAESOptimizerDeprecatedTest
> and the profile looking for the pattern ".*DeprecatedTest"?
>
> [Overall, I don't think it's worth the hassle given that it
> will/should
> be used only during a (hopefully short) transition period.]

+1 - provides more incentive to stabilize the API and get 4.0 out ;)

I do think we should try on a go forward basis to limit the run time
of tests and to look for opportunities to speed them up.  But I
never want to see that done at the expense of coverage /
completeness.  I tend to use -testOnly quite a bit during
development, but run the full suite before commits, which presently
takes me about 3.5 minutes on my not-exactly-wicked-fast macbook
(its a little faster when I run it inside a vbox ubuntu guest ;)

Phil
>
>
> Gilles
>
> [1] So that the usefulness of getting results of all tests with
> the same
>     basename is retained when refactoring.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Mime
View raw message