harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Geir Magnusson Jr." <g...@pobox.com>
Subject Re: [testing] test exclude list: can't we have incremental exclusions?
Date Sun, 26 Nov 2006 18:22:50 GMT


Alexei Zakharov wrote:
>> Sorry?  The 8 files reflect 8 *different platforms*, which are each
>> going to run anyway.
> 
> Right. You probably mean machine time here. But I was speaking about
> *human* time. For example if we had 5 different platforms and I 've
> ran my test only at one platform. In your scenario I need to:
> 1. Compose message to dev@list: "guys, could you please try this test
> on your favorite platform?" (timeWaste1)
> 2. "platform owner" reads this (timeWaste2) and takes over the task to
> run my test (timeWaste3)
> 3. after (2) he/she post the ACK or failure message to dev@list 
> (timeWaste4)
> 
> So we have four time wastes. Not a very big deal indeed. But if it
> isn't so much necessary couldn't we avoid it at least for some obvious
> cases?

Wouldn't the CI systems running on those platforms catch it automatically?

geir

> 
> Thanks,
> 
> 24.11.06, Geir Magnusson Jr.<geir@pobox.com> написал(а):
>>
>>
>> Alexei Zakharov wrote:
>> >> Could be - I don't see where the waste of time comes in.
>> >
>> > Just a simple arithmetic: 8 (test runs) is bigger (longer) than 1 or
>> > 2,  + synchronizing costs..
>>
>> Sorry?  The 8 files reflect 8 *different platforms*, which are each
>> going to run anyway.
>>
>> geir
>>
>>
>> >
>> > Thanks,
>> >
>> > 24.11.06, Geir Magnusson Jr.<geir@pobox.com> написал(а):
>> >>
>> >>
>> >> Alexei Zakharov wrote:
>> >> > In this case we can run against the situation when the test will
>> >> > remain excluded forever for some platform.
>> >>
>> >> Why?  Our goal is to get rid of any excluded tests.   Consider the
>> >> excluded tests as the existence of a regression, and we should work to
>> >> fix ASAP.
>> >>
>> >> We aren't "parking" them there to be ignored - we're putting them 
>> there
>> >> so that the build can complete while we work on the fixes.  We're in
>> >> essence "grandfathering-in" these regressions.
>> >>
>> >> > People have their own
>> >> > problems. IMHO we should do this only if there are suspicions 
>> that the
>> >> > test is platform dependent. Otherwise it is a waste of time IMO 
>> and CC
>> >> > can handle this.
>> >>
>> >> Could be - I don't see where the waste of time comes in.
>> >>
>> >> geir
>> >>
>> >> >
>> >> > Thanks,
>> >> >
>> >> > 24.11.06, Geir Magnusson Jr.<geir@pobox.com> написал(а):
>> >> >> Why not just remove from the platforms you have, and ask others
to
>> >> >> update (or not) platforms you don't have as appropriate?
>> >> >>
>> >> >> geir
>> >> >>
>> >> >> Tim Ellison wrote:
>> >> >> > Alexei Zakharov wrote:
>> >> >> >> Hi,
>> >> >> >>
>> >> >> >>> So we just should choice what is better: to break
sometimes
>> >> tests run
>> >> >> >>> or to
>> >> >> >>> forget enable test(s) on some platforms.
>> >> >> >> Yesterday, when I was removing one of the beans tests
from 
>> exclude
>> >> >> >> lists, I feel a bit uncomfortable while updating
>> >> >> >> exclude.linux.x86_64.xxx since I have no (easy) access
to such
>> >> systems
>> >> >> >> and had no plans to run tests on it. IMHO (in the perfect
world)
>> >> the
>> >> >> >> fact that I remove or add something from / to 
>> exclude.linux.x86_64
>> >> >> >> means I've at least ran tests for this platform and obtained

>> some
>> >> >> >> result. So let's have a common list,  it's easier to deal

>> with it
>> >> >> >> psychologically.  :-)
>> >> >> >>
>> >> >> >> As for above question: +1 for being optimists, i.e. to
remove
>> >> the test
>> >> >> >> from common list if it passes on all platforms available
to 
>> tester.
>> >> >> >
>> >> >> > +1 (otherwise I'll share the psychotherapy costs with you
:-)
> 
> 

Mime
View raw message