harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Ivanov" <ivavladi...@gmail.com>
Subject Re: [testing] test exclude list: can't we have incremental exclusions?
Date Fri, 24 Nov 2006 12:55:18 GMT
On 11/24/06, Alexei Zakharov <alexei.zakharov@gmail.com> wrote:
>
> Hi,
>
> > So we just should choice what is better: to break sometimes tests run or
> to
> > forget enable test(s) on some platforms.
>
> Yesterday, when I was removing one of the beans tests from exclude
> lists, I feel a bit uncomfortable while updating
> exclude.linux.x86_64.xxx since I have no (easy) access to such systems
> and had no plans to run tests on it. IMHO (in the perfect world) the
> fact that I remove or add something from / to exclude.linux.x86_64
> means I've at least ran tests for this platform and obtained some
> result. So let's have a common list,  it's easier to deal with it
> psychologically.  :-)


From my point of view, you should exclude test for x86 arch only and ask
somebody (through the dev list, for example) to verify it on x86_64.


As for above question: +1 for being optimists, i.e. to remove the test
> from common list if it passes on all platforms available to tester.


OK. I want to be optimistic too :)
So let's describe it on wiki(?) with some instruction what should be done in
case of test failure.

 Thanks, Vladimir

Thanks,
>
> 24.11.06, Vladimir Ivanov<ivavladimir@gmail.com> написал(а):
> > On 11/24/06, Ivanov, Alexey A <alexey.a.ivanov@intel.com> wrote:
> >
> > > Vladimir, all,
> > >
> > > These are good questions. On the other hand, there are reasons to
> consider
> > > most of the tests to be not platform dependent.
> > >
> > > If you fixed an issue, and it's in "all" exclude list, you would
> remove it
> > > from there and check on the platforms available to you. If something
> goes
> > > wrong on other platforms, those will report the problem back. Then you
> > > either fix the problem again, or add this test to the
> platform-specific
> > > exclude list.
> > >
> > > If you find some test fails, it's better to alert the community about
> the
> > > issue and file a JIRA issue. After the evaluation of the problem, a
> decision
> > > will be worked out how to exclude it: for all platforms or only for
> one
> > > specific, if the problem can't be easily fixed.
> > >
> > > Does it make sense?
> >
> >
> >
> > So we just should choice what is better: to break sometimes tests run or
> to
> > forget enable test(s) on some platforms.
> > My idea that is normal situation for ours exclude files is empty list.
> In
> > this case does not matter how many of them we have. But when we enable
> new
> > platform we use platform specific excludes.
> >
> > We may have any procedure to exclude/enable tests but it will nice if it
> > will one procedure for all :)
> >
> >
> >
> > > I am against duplication of the lists because it may easily hide a
> problem
> > > on other platform because someone forgot to update all 8 lists, for
> example.
> >
> >
> > I don't like duplication too but I think it is 'initial' state only. In
> the
> > nearest feature (I hope) we enable tests at least for win/lnx on x86 :)
> >
> >  Thanks, Vladimir
>
> --
> Alexei Zakharov,
> Intel Enterprise Solutions Software Division
>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message