harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivanov, Alexey A" <alexey.a.iva...@intel.com>
Subject RE: [testing] test exclude list: can't we have incremental exclusions?
Date Fri, 24 Nov 2006 13:28:43 GMT
>-----Original Message-----
>From: Vladimir Ivanov [mailto:ivavladimir@gmail.com]
>Sent: Friday, November 24, 2006 3:55 PM
>To: dev@harmony.apache.org
>Subject: Re: [testing] test exclude list: can't we have incremental
>exclusions?
>
>On 11/24/06, Alexei Zakharov <alexei.zakharov@gmail.com> wrote:
>>
>> Hi,
>>
>> > So we just should choice what is better: to break sometimes tests run
>or
>> to
>> > forget enable test(s) on some platforms.
>>
>> Yesterday, when I was removing one of the beans tests from exclude
>> lists, I feel a bit uncomfortable while updating
>> exclude.linux.x86_64.xxx since I have no (easy) access to such systems
>> and had no plans to run tests on it. IMHO (in the perfect world) the
>> fact that I remove or add something from / to exclude.linux.x86_64
>> means I've at least ran tests for this platform and obtained some
>> result. So let's have a common list,  it's easier to deal with it
>> psychologically.  :-)
>
>
>>From my point of view, you should exclude test for x86 arch only and ask
>somebody (through the dev list, for example) to verify it on x86_64.

I agree here.

>As for above question: +1 for being optimists, i.e. to remove the test
>> from common list if it passes on all platforms available to tester.

+1.
It's better to break build sometimes and face the problem ASAP rather than exclude a test
forever :)


And of course in the perfect world exclude lists will be empty for all modules. We should
strive for it.


Regards,
Alexey.

>OK. I want to be optimistic too :)
>So let's describe it on wiki(?) with some instruction what should be done
>in
>case of test failure.
>
> Thanks, Vladimir
>
>Thanks,
>>
>> 24.11.06, Vladimir Ivanov<ivavladimir@gmail.com> написал(а):
>> > On 11/24/06, Ivanov, Alexey A <alexey.a.ivanov@intel.com> wrote:
>> >
>> > > Vladimir, all,
>> > >
>> > > These are good questions. On the other hand, there are reasons to
>> consider
>> > > most of the tests to be not platform dependent.
>> > >
>> > > If you fixed an issue, and it's in "all" exclude list, you would
>> remove it
>> > > from there and check on the platforms available to you. If something
>> goes
>> > > wrong on other platforms, those will report the problem back. Then
>you
>> > > either fix the problem again, or add this test to the
>> platform-specific
>> > > exclude list.
>> > >
>> > > If you find some test fails, it's better to alert the community about
>> the
>> > > issue and file a JIRA issue. After the evaluation of the problem, a
>> decision
>> > > will be worked out how to exclude it: for all platforms or only for
>> one
>> > > specific, if the problem can't be easily fixed.
>> > >
>> > > Does it make sense?
>> >
>> >
>> >
>> > So we just should choice what is better: to break sometimes tests run
>or
>> to
>> > forget enable test(s) on some platforms.
>> > My idea that is normal situation for ours exclude files is empty list.
>> In
>> > this case does not matter how many of them we have. But when we enable
>> new
>> > platform we use platform specific excludes.
>> >
>> > We may have any procedure to exclude/enable tests but it will nice if
>it
>> > will one procedure for all :)
>> >
>> >
>> >
>> > > I am against duplication of the lists because it may easily hide a
>> problem
>> > > on other platform because someone forgot to update all 8 lists, for
>> example.
>> >
>> >
>> > I don't like duplication too but I think it is 'initial' state only. In
>> the
>> > nearest feature (I hope) we enable tests at least for win/lnx on x86 :)
>> >
>> >  Thanks, Vladimir
>>
>> --
>> Alexei Zakharov,
>> Intel Enterprise Solutions Software Division
>>

--
Alexey A. Ivanov
Intel Enterprise Solutions Software Division

Mime
View raw message