harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexei Zakharov" <alexei.zakha...@gmail.com>
Subject Re: [testing] test exclude list: can't we have incremental exclusions?
Date Fri, 24 Nov 2006 13:52:46 GMT
> From my point of view, you should exclude test for x86 arch only and ask
> somebody (through the dev list, for example) to verify it on x86_64

AFAIK Cruise Control was designed for this purpose, so if the test
fails on some platform everybody should get to know pretty soon. :)

Thanks,

24.11.06, Vladimir Ivanov<ivavladimir@gmail.com> написал(а):
> On 11/24/06, Alexei Zakharov <alexei.zakharov@gmail.com> wrote:
> >
> > Hi,
> >
> > > So we just should choice what is better: to break sometimes tests run or
> > to
> > > forget enable test(s) on some platforms.
> >
> > Yesterday, when I was removing one of the beans tests from exclude
> > lists, I feel a bit uncomfortable while updating
> > exclude.linux.x86_64.xxx since I have no (easy) access to such systems
> > and had no plans to run tests on it. IMHO (in the perfect world) the
> > fact that I remove or add something from / to exclude.linux.x86_64
> > means I've at least ran tests for this platform and obtained some
> > result. So let's have a common list,  it's easier to deal with it
> > psychologically.  :-)
>
>
> From my point of view, you should exclude test for x86 arch only and ask
> somebody (through the dev list, for example) to verify it on x86_64.
>
>
> As for above question: +1 for being optimists, i.e. to remove the test
> > from common list if it passes on all platforms available to tester.
>
>
> OK. I want to be optimistic too :)
> So let's describe it on wiki(?) with some instruction what should be done in
> case of test failure.
>
>  Thanks, Vladimir
>
> Thanks,
> >
> > 24.11.06, Vladimir Ivanov<ivavladimir@gmail.com> написал(а):
> > > On 11/24/06, Ivanov, Alexey A <alexey.a.ivanov@intel.com> wrote:
> > >
> > > > Vladimir, all,
> > > >
> > > > These are good questions. On the other hand, there are reasons to
> > consider
> > > > most of the tests to be not platform dependent.
> > > >
> > > > If you fixed an issue, and it's in "all" exclude list, you would
> > remove it
> > > > from there and check on the platforms available to you. If something
> > goes
> > > > wrong on other platforms, those will report the problem back. Then you
> > > > either fix the problem again, or add this test to the
> > platform-specific
> > > > exclude list.
> > > >
> > > > If you find some test fails, it's better to alert the community about
> > the
> > > > issue and file a JIRA issue. After the evaluation of the problem, a
> > decision
> > > > will be worked out how to exclude it: for all platforms or only for
> > one
> > > > specific, if the problem can't be easily fixed.
> > > >
> > > > Does it make sense?
> > >
> > >
> > >
> > > So we just should choice what is better: to break sometimes tests run or
> > to
> > > forget enable test(s) on some platforms.
> > > My idea that is normal situation for ours exclude files is empty list.
> > In
> > > this case does not matter how many of them we have. But when we enable
> > new
> > > platform we use platform specific excludes.
> > >
> > > We may have any procedure to exclude/enable tests but it will nice if it
> > > will one procedure for all :)
> > >
> > >
> > >
> > > > I am against duplication of the lists because it may easily hide a
> > problem
> > > > on other platform because someone forgot to update all 8 lists, for
> > example.
> > >
> > >
> > > I don't like duplication too but I think it is 'initial' state only. In
> > the
> > > nearest feature (I hope) we enable tests at least for win/lnx on x86 :)



-- 
Alexei Zakharov,
Intel Enterprise Solutions Software Division
Mime
View raw message