harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexei Zakharov" <alexei.zakha...@gmail.com>
Subject Re: [testing] test exclude list: can't we have incremental exclusions?
Date Sun, 26 Nov 2006 10:46:47 GMT
> Sorry?  The 8 files reflect 8 *different platforms*, which are each
> going to run anyway.

Right. You probably mean machine time here. But I was speaking about
*human* time. For example if we had 5 different platforms and I 've
ran my test only at one platform. In your scenario I need to:
1. Compose message to dev@list: "guys, could you please try this test
on your favorite platform?" (timeWaste1)
2. "platform owner" reads this (timeWaste2) and takes over the task to
run my test (timeWaste3)
3. after (2) he/she post the ACK or failure message to dev@list (timeWaste4)

So we have four time wastes. Not a very big deal indeed. But if it
isn't so much necessary couldn't we avoid it at least for some obvious
cases?

Thanks,

24.11.06, Geir Magnusson Jr.<geir@pobox.com> написал(а):
>
>
> Alexei Zakharov wrote:
> >> Could be - I don't see where the waste of time comes in.
> >
> > Just a simple arithmetic: 8 (test runs) is bigger (longer) than 1 or
> > 2,  + synchronizing costs..
>
> Sorry?  The 8 files reflect 8 *different platforms*, which are each
> going to run anyway.
>
> geir
>
>
> >
> > Thanks,
> >
> > 24.11.06, Geir Magnusson Jr.<geir@pobox.com> написал(а):
> >>
> >>
> >> Alexei Zakharov wrote:
> >> > In this case we can run against the situation when the test will
> >> > remain excluded forever for some platform.
> >>
> >> Why?  Our goal is to get rid of any excluded tests.   Consider the
> >> excluded tests as the existence of a regression, and we should work to
> >> fix ASAP.
> >>
> >> We aren't "parking" them there to be ignored - we're putting them there
> >> so that the build can complete while we work on the fixes.  We're in
> >> essence "grandfathering-in" these regressions.
> >>
> >> > People have their own
> >> > problems. IMHO we should do this only if there are suspicions that the
> >> > test is platform dependent. Otherwise it is a waste of time IMO and CC
> >> > can handle this.
> >>
> >> Could be - I don't see where the waste of time comes in.
> >>
> >> geir
> >>
> >> >
> >> > Thanks,
> >> >
> >> > 24.11.06, Geir Magnusson Jr.<geir@pobox.com> написал(а):
> >> >> Why not just remove from the platforms you have, and ask others to
> >> >> update (or not) platforms you don't have as appropriate?
> >> >>
> >> >> geir
> >> >>
> >> >> Tim Ellison wrote:
> >> >> > Alexei Zakharov wrote:
> >> >> >> Hi,
> >> >> >>
> >> >> >>> So we just should choice what is better: to break sometimes
> >> tests run
> >> >> >>> or to
> >> >> >>> forget enable test(s) on some platforms.
> >> >> >> Yesterday, when I was removing one of the beans tests from
exclude
> >> >> >> lists, I feel a bit uncomfortable while updating
> >> >> >> exclude.linux.x86_64.xxx since I have no (easy) access to
such
> >> systems
> >> >> >> and had no plans to run tests on it. IMHO (in the perfect
world)
> >> the
> >> >> >> fact that I remove or add something from / to exclude.linux.x86_64
> >> >> >> means I've at least ran tests for this platform and obtained
some
> >> >> >> result. So let's have a common list,  it's easier to deal
with it
> >> >> >> psychologically.  :-)
> >> >> >>
> >> >> >> As for above question: +1 for being optimists, i.e. to remove
> >> the test
> >> >> >> from common list if it passes on all platforms available to
tester.
> >> >> >
> >> >> > +1 (otherwise I'll share the psychotherapy costs with you :-)


-- 
Alexei Zakharov,
Intel Enterprise Solutions Software Division
Mime
View raw message