harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexei Zakharov" <alexei.zakha...@gmail.com>
Subject Re: [testing] test exclude list: can't we have incremental exclusions?
Date Fri, 24 Nov 2006 12:36:44 GMT

> So we just should choice what is better: to break sometimes tests run or to
> forget enable test(s) on some platforms.

Yesterday, when I was removing one of the beans tests from exclude
lists, I feel a bit uncomfortable while updating
exclude.linux.x86_64.xxx since I have no (easy) access to such systems
and had no plans to run tests on it. IMHO (in the perfect world) the
fact that I remove or add something from / to exclude.linux.x86_64
means I've at least ran tests for this platform and obtained some
result. So let's have a common list,  it's easier to deal with it
psychologically.  :-)

As for above question: +1 for being optimists, i.e. to remove the test
from common list if it passes on all platforms available to tester.


24.11.06, Vladimir Ivanov<ivavladimir@gmail.com> написал(а):
> On 11/24/06, Ivanov, Alexey A <alexey.a.ivanov@intel.com> wrote:
> > Vladimir, all,
> >
> > These are good questions. On the other hand, there are reasons to consider
> > most of the tests to be not platform dependent.
> >
> > If you fixed an issue, and it's in "all" exclude list, you would remove it
> > from there and check on the platforms available to you. If something goes
> > wrong on other platforms, those will report the problem back. Then you
> > either fix the problem again, or add this test to the platform-specific
> > exclude list.
> >
> > If you find some test fails, it's better to alert the community about the
> > issue and file a JIRA issue. After the evaluation of the problem, a decision
> > will be worked out how to exclude it: for all platforms or only for one
> > specific, if the problem can't be easily fixed.
> >
> > Does it make sense?
> So we just should choice what is better: to break sometimes tests run or to
> forget enable test(s) on some platforms.
> My idea that is normal situation for ours exclude files is empty list. In
> this case does not matter how many of them we have. But when we enable new
> platform we use platform specific excludes.
> We may have any procedure to exclude/enable tests but it will nice if it
> will one procedure for all :)
> > I am against duplication of the lists because it may easily hide a problem
> > on other platform because someone forgot to update all 8 lists, for example.
> I don't like duplication too but I think it is 'initial' state only. In the
> nearest feature (I hope) we enable tests at least for win/lnx on x86 :)
>  Thanks, Vladimir

Alexei Zakharov,
Intel Enterprise Solutions Software Division
View raw message