harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Geir Magnusson Jr." <g...@pobox.com>
Subject Re: svn commit: r477149 - in /harmony/enhanced/drlvm/trunk/build/make: targets/kernel.test.xml test.properties
Date Tue, 21 Nov 2006 04:00:15 GMT


Alexey Varlamov wrote:
> 2006/11/21, Gregory Shimansky <gshimansky@gmail.com>:
>> Geir Magnusson Jr. wrote:
>> >
>> >
>> > Alexey Varlamov wrote:
>> >> Folks,
>> >>
>> >> I've just did a little step in improving pre-commit testing for DRLVM;
>> >> most important change is a move from "perTest" forking mode to "once"
>> >> (aka sameVM mode).
>> >> This reduces testing time drastically (~50%), but may introduce some
>> >> extra instability (like new intermittent failures or timeouts).
>> >
>> > Plus the inability to figure out what's screwing things up.
>> >
>> > I can't decide if I like this.  On one hand, I like it because it's
>> > actually better to find side-effects - some test can pass, but still
>> > leave the VM in a broken state that another test will show.
>>
>> I think the change has a good intension but it should've gone through
>> more testing. Preferably the fixes for problems on windows should've
>> been included into the patch. If the problems are too hard to fix, why
>> not open a discussion before changing the files in SVN?
> 
> I did test on SUSE9, Win2003 server and WinXP(laptop) - only spotted
> *intermittent* failures on WinXP. I assumed it maybe little bothering
> but really easy to switch; OTOH we better discover instabilities
> sooner if they are frequent.
> 
>>
>> > OTOH, it does remove the clarity of each test being a single, separate
>> > test.  It conceptually mixes integration testing with unit testing.
>> >
>> > Can you please just add a switch?  That way in the event of a failure,
>> > we can re-run with forking on, and therefore can tell if the crash is
>> > specifically due to the test that is crashing, or a side effect caused
>> > by something that came before.
>>
>> I would also like a switch. I don't like it that a change which
>> knowingly introduces problems with acceptance tests is committed without
>> discussion. Should we discard kernel tests from drlvm commits until they
>> are fixed? I think that there was an agreement about no regression. So
>> now all commits to drlvm are blocked.
> 
> But there *is* cmd-line switch, my bad that I didn't explained this
> more clearly. And I'm not sure we firmly adopted "zero regression"
> policy - at least intermittent failures in kernel tests did block
> nobody so far. Should we vote formally, BTW?

No.  We've always been zero regression in classlib - we've never been 
clean enough with DRLVM to even know it.

So do I don't think we should make formal votes about - lets just drive 
forward to make it so.

> Generally I agree it is always better to discuss, and in fact
> hesitated to commit with default forking set to "once" - but decided
> this is right thing to do rigth now. The current trend to make drlvm
> truly standard VM for harmony, recently Alexei even proclaimed 100%
> HUT pass rate, neverthless drlvm is not yet reliable enough and we
> should keep the momentum and face possible problems with open eyes.
> Again, I'm sorry to bother you^H^H^H us VM guys, but believe this is
> neccessary evil now.

The nicer way to do this is to introduce this change as an option, and 
let people start using it...

geir


Mime
View raw message