harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivanov, Alexey A" <alexey.a.iva...@intel.com>
Subject RE: [testing] test exclude list: can't we have incremental exclusions?
Date Fri, 24 Nov 2006 07:44:22 GMT
Vladimir, all,

These are good questions. On the other hand, there are reasons to consider most of the tests
to be not platform dependent.

If you fixed an issue, and it's in "all" exclude list, you would remove it from there and
check on the platforms available to you. If something goes wrong on other platforms, those
will report the problem back. Then you either fix the problem again, or add this test to the
platform-specific exclude list.

If you find some test fails, it's better to alert the community about the issue and file a
JIRA issue. After the evaluation of the problem, a decision will be worked out how to exclude
it: for all platforms or only for one specific, if the problem can't be easily fixed.

Does it make sense?


I am against duplication of the lists because it may easily hide a problem on other platform
because someone forgot to update all 8 lists, for example.


Regards,
--
Alexey A. Ivanov
Intel Enterprise Solutions Software Division


>-----Original Message-----
>From: Vladimir Ivanov [mailto:ivavladimir@gmail.com]
>Sent: Friday, November 24, 2006 9:09 AM
>To: dev@harmony.apache.org
>Subject: Re: [testing] test exclude list: can't we have incremental
>exclusions?
>
>Actually, 8 files for each module :)
>
>
>
>I do it purposely to clean up exclude lists easily. When the test is
>excluded for one platform it should be run on one platform only to be sure
>that it is enabled. If we have 'common' part than to enable test it should
>be run on all available platforms.
>
>
>
>I believe not all community members have at least windows and Linux for x86
>and x86_64 and not all of them plan to by pair of Itanium boxes :)
>
>
>
>So, the question is: should I check the update on *all* available platforms
>(one exclude list) or only on *available to me* (separated exclude lists)?
>
>Also, if the test is failed for me should I exclude it in the common or
>specific part of exclude list?
>
>
>
>May be we write somewhere answer for these questions to understand the test
>excluding/ enabling procedure?
>
>
>
> Thanks, Vladimir
>
>
>
>On 11/24/06, Alexey Petrenko <alexey.a.petrenko@gmail.com> wrote:
>>
>> Good idea, Geir.
>>
>> SY, Alexey
>>
>> 23.11.06, Geir Magnusson Jr.<geir@pobox.com> написал(а):
>> > And while you're at it, how about making kind-and-gentle support for
>> > local excludes such that I can have a file
>> >
>> >     exclude.local
>> >
>> > which is my local exclusion list that
>> >
>> > a) will be svn-ignored and
>> >
>> > b) doesn't have to be there - so if a developer hasn't created the file,
>> > the build just keeps going...  I *think* that not having the file for
>an
>> > <excludesfile> entry will let the build keep going, but I'm not sure.
>> >
>> > geir
>> >
>> >
>> > Geir Magnusson Jr. wrote:
>> > > That works for me.  It will only increase the number of files if
>> > > platforms have bugs, but it will make for easier maintenance.
>> > >
>> > > We'll do the same in DRLVM too.
>> > >
>> > > geir
>> > >
>> > > Ivanov, Alexey A wrote:
>> > >> Hi everyone,
>> > >>
>> > >> Recently test exclude lists were removed from build.xml of the
>> > >> corresponding module, and there were added *six* files with excluded
>> > >> tests. These files contain almost the same list of files. The lists
>> > >> are identical for swing module. I found 2 differences for awt module
>> > >> (there are still about 50 files names listed in every of the exclude
>> > >> lists).
>> > >>
>> > >> Why can't we use one 'exclude.all' file to exclude tests which fail
>> on
>> > >> every platform? It's an obvious optimization.
>> > >> I've tested the approach of using several exclude list files on
>> > >> build.xml of swing module. It works just fine.
>> > >>
>> > >> Your comments?
>> > >>
>> > >> Regards,
>> > >> Alexey.
>> > >>
>> > >>
>> > >> ----- build.xml patch --------
>> > >> Index: build.xml
>> > >> ===================================================================
>> > >> --- build.xml   (revision 478584)
>> > >> +++ build.xml   (working copy)
>> > >> @@ -186,6 +186,7 @@
>> > >>
>> > >>                  <fileset
>dir="${hy.swing.src.test.api}/java/common">
>> > >>                      <include name="**/*Test*.java"/>
>> > >> +                       <excludesfile name="./make/exclude.all"
/>
>> > >>                      <excludesfile name="${exclude.file}" />
>> > >>                  </fileset>
>> > >>              </batchtest>
>> > >> ------------------------------
>> > >>
>> > >>
>> > >> --
>> > >> Alexey A. Ivanov
>> > >> Intel Enterprise Solutions Software Division
>> >
>>

Mime
View raw message