harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexey Varlamov" <alexey.v.varla...@gmail.com>
Subject Re: svn commit: r477149 - in /harmony/enhanced/drlvm/trunk/build/make: targets/kernel.test.xml test.properties
Date Tue, 21 Nov 2006 03:46:00 GMT
2006/11/21, Gregory Shimansky <gshimansky@gmail.com>:
> Geir Magnusson Jr. wrote:
> >
> >
> > Alexey Varlamov wrote:
> >> Folks,
> >>
> >> I've just did a little step in improving pre-commit testing for DRLVM;
> >> most important change is a move from "perTest" forking mode to "once"
> >> (aka sameVM mode).
> >> This reduces testing time drastically (~50%), but may introduce some
> >> extra instability (like new intermittent failures or timeouts).
> >
> > Plus the inability to figure out what's screwing things up.
> >
> > I can't decide if I like this.  On one hand, I like it because it's
> > actually better to find side-effects - some test can pass, but still
> > leave the VM in a broken state that another test will show.
>
> I think the change has a good intension but it should've gone through
> more testing. Preferably the fixes for problems on windows should've
> been included into the patch. If the problems are too hard to fix, why
> not open a discussion before changing the files in SVN?

I did test on SUSE9, Win2003 server and WinXP(laptop) - only spotted
*intermittent* failures on WinXP. I assumed it maybe little bothering
but really easy to switch; OTOH we better discover instabilities
sooner if they are frequent.

>
> > OTOH, it does remove the clarity of each test being a single, separate
> > test.  It conceptually mixes integration testing with unit testing.
> >
> > Can you please just add a switch?  That way in the event of a failure,
> > we can re-run with forking on, and therefore can tell if the crash is
> > specifically due to the test that is crashing, or a side effect caused
> > by something that came before.
>
> I would also like a switch. I don't like it that a change which
> knowingly introduces problems with acceptance tests is committed without
> discussion. Should we discard kernel tests from drlvm commits until they
> are fixed? I think that there was an agreement about no regression. So
> now all commits to drlvm are blocked.

But there *is* cmd-line switch, my bad that I didn't explained this
more clearly. And I'm not sure we firmly adopted "zero regression"
policy - at least intermittent failures in kernel tests did block
nobody so far. Should we vote formally, BTW?
Generally I agree it is always better to discuss, and in fact
hesitated to commit with default forking set to "once" - but decided
this is right thing to do rigth now. The current trend to make drlvm
truly standard VM for harmony, recently Alexei even proclaimed 100%
HUT pass rate, neverthless drlvm is not yet reliable enough and we
should keep the momentum and face possible problems with open eyes.
Again, I'm sorry to bother you^H^H^H us VM guys, but believe this is
neccessary evil now.

--
Alexey

Mime
View raw message