harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Ivanov" <ivavladi...@gmail.com>
Subject Re: [testing] test exclude list: can't we have incremental exclusions?
Date Fri, 24 Nov 2006 09:45:01 GMT
On 11/24/06, Ivanov, Alexey A <alexey.a.ivanov@intel.com> wrote:

> Vladimir, all,
>
> These are good questions. On the other hand, there are reasons to consider
> most of the tests to be not platform dependent.
>
> If you fixed an issue, and it's in "all" exclude list, you would remove it
> from there and check on the platforms available to you. If something goes
> wrong on other platforms, those will report the problem back. Then you
> either fix the problem again, or add this test to the platform-specific
> exclude list.
>
> If you find some test fails, it's better to alert the community about the
> issue and file a JIRA issue. After the evaluation of the problem, a decision
> will be worked out how to exclude it: for all platforms or only for one
> specific, if the problem can't be easily fixed.
>
> Does it make sense?



So we just should choice what is better: to break sometimes tests run or to
forget enable test(s) on some platforms.
My idea that is normal situation for ours exclude files is empty list. In
this case does not matter how many of them we have. But when we enable new
platform we use platform specific excludes.

We may have any procedure to exclude/enable tests but it will nice if it
will one procedure for all :)



> I am against duplication of the lists because it may easily hide a problem
> on other platform because someone forgot to update all 8 lists, for example.


I don't like duplication too but I think it is 'initial' state only. In the
nearest feature (I hope) we enable tests at least for win/lnx on x86 :)

 Thanks, Vladimir





> Regards,
> --
> Alexey A. Ivanov
> Intel Enterprise Solutions Software Division
>
>
> >-----Original Message-----
> >From: Vladimir Ivanov [mailto:ivavladimir@gmail.com]
> >Sent: Friday, November 24, 2006 9:09 AM
> >To: dev@harmony.apache.org
> >Subject: Re: [testing] test exclude list: can't we have incremental
> >exclusions?
> >
> >Actually, 8 files for each module :)
> >
> >
> >
> >I do it purposely to clean up exclude lists easily. When the test is
> >excluded for one platform it should be run on one platform only to be
> sure
> >that it is enabled. If we have 'common' part than to enable test it
> should
> >be run on all available platforms.
> >
> >
> >
> >I believe not all community members have at least windows and Linux for
> x86
> >and x86_64 and not all of them plan to by pair of Itanium boxes :)
> >
> >
> >
> >So, the question is: should I check the update on *all* available
> platforms
> >(one exclude list) or only on *available to me* (separated exclude
> lists)?
> >
> >Also, if the test is failed for me should I exclude it in the common or
> >specific part of exclude list?
> >
> >
> >
> >May be we write somewhere answer for these questions to understand the
> test
> >excluding/ enabling procedure?
> >
> >
> >
> > Thanks, Vladimir
> >
> >
> >
> >On 11/24/06, Alexey Petrenko <alexey.a.petrenko@gmail.com> wrote:
> >>
> >> Good idea, Geir.
> >>
> >> SY, Alexey
> >>
> >> 23.11.06, Geir Magnusson Jr.<geir@pobox.com> написал(а):
> >> > And while you're at it, how about making kind-and-gentle support for
> >> > local excludes such that I can have a file
> >> >
> >> >     exclude.local
> >> >
> >> > which is my local exclusion list that
> >> >
> >> > a) will be svn-ignored and
> >> >
> >> > b) doesn't have to be there - so if a developer hasn't created the
> file,
> >> > the build just keeps going...  I *think* that not having the file for
> >an
> >> > <excludesfile> entry will let the build keep going, but I'm not sure.
> >> >
> >> > geir
> >> >
> >> >
> >> > Geir Magnusson Jr. wrote:
> >> > > That works for me.  It will only increase the number of files if
> >> > > platforms have bugs, but it will make for easier maintenance.
> >> > >
> >> > > We'll do the same in DRLVM too.
> >> > >
> >> > > geir
> >> > >
> >> > > Ivanov, Alexey A wrote:
> >> > >> Hi everyone,
> >> > >>
> >> > >> Recently test exclude lists were removed from build.xml of the
> >> > >> corresponding module, and there were added *six* files with
> excluded
> >> > >> tests. These files contain almost the same list of files. The
> lists
> >> > >> are identical for swing module. I found 2 differences for awt
> module
> >> > >> (there are still about 50 files names listed in every of the
> exclude
> >> > >> lists).
> >> > >>
> >> > >> Why can't we use one 'exclude.all' file to exclude tests which
> fail
> >> on
> >> > >> every platform? It's an obvious optimization.
> >> > >> I've tested the approach of using several exclude list files on
> >> > >> build.xml of swing module. It works just fine.
> >> > >>
> >> > >> Your comments?
> >> > >>
> >> > >> Regards,
> >> > >> Alexey.
> >> > >>
> >> > >>
> >> > >> ----- build.xml patch --------
> >> > >> Index: build.xml
> >> > >>
> ===================================================================
> >> > >> --- build.xml   (revision 478584)
> >> > >> +++ build.xml   (working copy)
> >> > >> @@ -186,6 +186,7 @@
> >> > >>
> >> > >>                  <fileset
> >dir="${hy.swing.src.test.api}/java/common">
> >> > >>                      <include name="**/*Test*.java"/>
> >> > >> +                       <excludesfile name="./make/exclude.all"
/>
> >> > >>                      <excludesfile name="${exclude.file}" />
> >> > >>                  </fileset>
> >> > >>              </batchtest>
> >> > >> ------------------------------
> >> > >>
> >> > >>
> >> > >> --
> >> > >> Alexey A. Ivanov
> >> > >> Intel Enterprise Solutions Software Division
> >> >
> >>
>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message