harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexey Varlamov" <alexey.v.varla...@gmail.com>
Subject Re: svn commit: r477149 - in /harmony/enhanced/drlvm/trunk/build/make: targets/kernel.test.xml test.properties
Date Tue, 21 Nov 2006 04:14:42 GMT
2006/11/21, Alexei Fedotov <alexei.fedotov@gmail.com>:
> Hello,
> If a developer runs all tests in one VM and they pass, that's great
> and he can commit. Can developer make a commit if tests pass in
> "perTest" mode and fail in "once" mode?
> Let's imagine following scenarios of cooperative development when
> pre-commit test suite demonstrates unpredictable behavior (such as
> "once" mode").
> Scenario A.
> 1. A developer runs tests one time, and tests failed.
> 2. He runs them again and tests passes.
> 3. He commit changes.
> Why do we need to exercise developers' patience? Simple tests are
> predictable - they either pass or fail.
> Scenario B.
> 1. Developer A pass tests and commit his changes.
> 2. Developer B checks out a fresh workspace, makes changes and runs
> tests - they failed.
> 3. He digs out the problem to the stage when he runs tests for the
> clean workspace - tests fail since he has slightly different
> configuration.
> Developer B suspects developer A to run tests multiple times, or skip
> acceptance test run at all. This destroys mutual trust. This makes me
> think that the simpler pre-commit tests are, the better.

So in extreme case, no testing at all is the best! ;)
Seriously, kernel tests are simple enough unit tests - if we cannot
even trust "sameVM" runs here, do we ever need to invent complex
testing scenarios involving big apps?

Sure, we can reverse the default to "perTest" mode, but won't you feel
deeply sorry for time wasted in such testing? Just to remind, tests
for debug version take up to 2h on Windows server.

> Thanks!

View raw message