harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Ivanov" <ivavladi...@gmail.com>
Subject Re: [classlib][build] exclude list impl issues
Date Mon, 29 Jan 2007 08:45:32 GMT
On 1/29/07, Geir Magnusson Jr. <geir@pobox.com> wrote:
>
>
> On Jan 28, 2007, at 11:28 PM, Vladimir Ivanov wrote:
>
> > OK. Current changes for exclude lists were integrated and now we have
> > 3-level exclude list:
> >
> > exlude.common - tests, that failed over all platforms
> >
> > exclude.<platform> - tests, that failed over specific platform only
> >
> > exclude.<platform>.interm - tests that failed time to time over
> > specific
> > platform only.
>
> Quick q - why separate interm out?  why not just put in the platform
> file?


To do exclude lists clean up process easier. To delete test from the
platform list you should check that this test passed one time. To delete
test from 'intermittently failed' exclude list each test should be run in
cycle.

>
> > Any file in this chain may be skipped. Final exclude lists are
> > generated at
> > the build time and stored to the ${hy.hdk}/build directory. Thanks
> > to Alexei
> > Zakharov for this changes.
>
> Another quick q - why not just glom things together in memory?



If we will run tests against HDK it will nice to skip excluded tests. For
this reason exclude lists should be stored somewhere in the built product.

>
> >
> >
> > But I have one more question to discuss: should we use the tests from
> > 'intermittently failed' exclude list for preintegration testing?
>
> What does that mean?



OK. Now pre-integration test include 'ant test' with requirements that all
tests should be passed. While we have intermittently failed tests this
target sometimes report 'failed' status for these tests without correlation
with commit changes. We have 2 options here:

- exclude these tests in the platform-specific ex-list

- exclude these tests in the special ex-list

Each option has pro and contra: in the first case we can miss the regression
when intermittently failed test became always failed or delete test from
exclude lists if it passed only one time.

In the second case we have some overhead to support special exclude lists.

Note to detect regression the 'default' mode should be 'on (run)
intermittently failed tests' to test commit changes and 'off' to run tests
under CC.

I'll add switches to on/off levels for resulting exclude lists.

 thanks, Vladimir



> > If we use
> > it we may miss some regression when intermittently failed test will
> > failed
> > constantly however if we does not use it we need to run test twice
> > sometimes.
> > What is correct behavior?
> >
> >  thanks, Vladimir
> >
> >
> >
> > On 1/28/07, Alex Blewitt <alex.blewitt@gmail.com> wrote:
> >>
> >> Yeah, +1 for using common exclude lists. It makes it easier when
> >> Harmony gets ported to other operating systems. And I don't see the
> >> benefit of having empty lists in that case; and if nothing's failing,
> >> you don't need an empty list either :-)
> >>
> >> Alex.
> >>
> >> On 28/01/07, Alexey Petrenko <alexey.a.petrenko@gmail.com> wrote:
> >> > +1 from me for using common exclude lists and removing empty lists.
> >> >
> >> > SY, Alexey
> >> >
> >> > 2007/1/16, Alexey Varlamov <alexey.v.varlamov@gmail.com>:
> >> > > Folks,
> >> > >
> >> > > I've some concerns with recent updates for x-list support: can
> >> we use
> >> > > a common macros instead of copy-pasting 4 new targets into
> >> build.xml
> >> > > for each module?
> >> > > Or, if we can neglect creating a compiled x-file
> >> > > (${hy.hdk}/build/<module>.exclude), just use "if" attribute
of
> >> > > <excludesfile>, like this:
> >> > >
> >> > > <available property="x.list.exist"
> >> > > file="exclude.${hy.platform}.${hy.test.vm.name}"/>
> >> > > ...
> >> > >             <batchtest>
> >> > >                 <fileset dir="${src.test.java}">
> >> > >                     <include name="**/*Test.java"/>
> >> > >                     <excludesfile name="exclude.common"/>
> >> > >                     <excludesfile name="${exclude.file} if="
> >> x.list.exist" />
> >> > >                 </fileset>
> >> > >             </batchtest>
> >> > >
> >> > > Also, I suggest to delete empty x-list remained after introducing
> >> common lists.
> >> > >
> >> > > Another issue is with "hy.test.vm.name", it was a surprise for
> >> me that
> >> > > it is not autodetected yet. Most obvious way to get it is to read
> >> > > "java.vm.name" property, this only requires running trivial test.
> >> > >
> >> > > --
> >> > > Alexey
> >> > >
> >> >
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message