harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mikhail Loenko" <mloe...@gmail.com>
Subject Re: [classlib][build] exclude list impl issues
Date Tue, 30 Jan 2007 05:17:27 GMT
2007/1/29, Geir Magnusson Jr. <geir@pobox.com>:
>
> On Jan 29, 2007, at 8:45 AM, Vladimir Ivanov wrote:
>
> > On 1/29/07, Geir Magnusson Jr. <geir@pobox.com> wrote:
> >>
> >>
> >> On Jan 28, 2007, at 11:28 PM, Vladimir Ivanov wrote:
> >>
> >> > OK. Current changes for exclude lists were integrated and now we
> >> have
> >> > 3-level exclude list:
> >> >
> >> > exlude.common - tests, that failed over all platforms
> >> >
> >> > exclude.<platform> - tests, that failed over specific platform only
> >> >
> >> > exclude.<platform>.interm - tests that failed time to time over
> >> > specific
> >> > platform only.
> >>
> >> Quick q - why separate interm out?  why not just put in the platform
> >> file?
> >
> >
> > To do exclude lists clean up process easier. To delete test from the
> > platform list you should check that this test passed one time. To
> > delete
> > test from 'intermittently failed' exclude list each test should be
> > run in
> > cycle.
>
> :/
>
> Doesn't the first case simply mean you got lucky?

Well if some permanently failing tests starts to pass then it's
probably more than
just luck

I'm for separating these x-lists: we may get rid of CC failures, and
at the same time
those tests will still find regressions (when run pre-commit tests)

Thanks,
Mikhail


>
> >
> >>
> >> > Any file in this chain may be skipped. Final exclude lists are
> >> > generated at
> >> > the build time and stored to the ${hy.hdk}/build directory. Thanks
> >> > to Alexei
> >> > Zakharov for this changes.
> >>
> >> Another quick q - why not just glom things together in memory?
> >
> >
> >
> > If we will run tests against HDK it will nice to skip excluded
> > tests. For
> > this reason exclude lists should be stored somewhere in the built
> > product.
> >
>
> That makes perfect sense.  Thanks
>
> geir
>
> >>
> >> >
> >> >
> >> > But I have one more question to discuss: should we use the tests
> >> from
> >> > 'intermittently failed' exclude list for preintegration testing?
> >>
> >> What does that mean?
> >
> >
> >
> > OK. Now pre-integration test include 'ant test' with requirements
> > that all
> > tests should be passed. While we have intermittently failed tests this
> > target sometimes report 'failed' status for these tests without
> > correlation
> > with commit changes. We have 2 options here:
> >
> > - exclude these tests in the platform-specific ex-list
> >
> > - exclude these tests in the special ex-list
> >
> > Each option has pro and contra: in the first case we can miss the
> > regression
> > when intermittently failed test became always failed or delete test
> > from
> > exclude lists if it passed only one time.
> >
> > In the second case we have some overhead to support special exclude
> > lists.
> >
> > Note to detect regression the 'default' mode should be 'on (run)
> > intermittently failed tests' to test commit changes and 'off' to
> > run tests
> > under CC.
> >
> > I'll add switches to on/off levels for resulting exclude lists.
> >
> > thanks, Vladimir
> >
> >
> >
> >> > If we use
> >> > it we may miss some regression when intermittently failed test will
> >> > failed
> >> > constantly however if we does not use it we need to run test twice
> >> > sometimes.
> >> > What is correct behavior?
> >> >
> >> >  thanks, Vladimir
> >> >
> >> >
> >> >
> >> > On 1/28/07, Alex Blewitt <alex.blewitt@gmail.com> wrote:
> >> >>
> >> >> Yeah, +1 for using common exclude lists. It makes it easier when
> >> >> Harmony gets ported to other operating systems. And I don't see
> >> the
> >> >> benefit of having empty lists in that case; and if nothing's
> >> failing,
> >> >> you don't need an empty list either :-)
> >> >>
> >> >> Alex.
> >> >>
> >> >> On 28/01/07, Alexey Petrenko <alexey.a.petrenko@gmail.com> wrote:
> >> >> > +1 from me for using common exclude lists and removing empty
> >> lists.
> >> >> >
> >> >> > SY, Alexey
> >> >> >
> >> >> > 2007/1/16, Alexey Varlamov <alexey.v.varlamov@gmail.com>:
> >> >> > > Folks,
> >> >> > >
> >> >> > > I've some concerns with recent updates for x-list support:
can
> >> >> we use
> >> >> > > a common macros instead of copy-pasting 4 new targets into
> >> >> build.xml
> >> >> > > for each module?
> >> >> > > Or, if we can neglect creating a compiled x-file
> >> >> > > (${hy.hdk}/build/<module>.exclude), just use "if" attribute
of
> >> >> > > <excludesfile>, like this:
> >> >> > >
> >> >> > > <available property="x.list.exist"
> >> >> > > file="exclude.${hy.platform}.${hy.test.vm.name}"/>
> >> >> > > ...
> >> >> > >             <batchtest>
> >> >> > >                 <fileset dir="${src.test.java}">
> >> >> > >                     <include name="**/*Test.java"/>
> >> >> > >                     <excludesfile name="exclude.common"/>
> >> >> > >                     <excludesfile name="${exclude.file}
if="
> >> >> x.list.exist" />
> >> >> > >                 </fileset>
> >> >> > >             </batchtest>
> >> >> > >
> >> >> > > Also, I suggest to delete empty x-list remained after
> >> introducing
> >> >> common lists.
> >> >> > >
> >> >> > > Another issue is with "hy.test.vm.name", it was a surprise
for
> >> >> me that
> >> >> > > it is not autodetected yet. Most obvious way to get it is
> >> to read
> >> >> > > "java.vm.name" property, this only requires running trivial
> >> test.
> >> >> > >
> >> >> > > --
> >> >> > > Alexey
> >> >> > >
> >> >> >
> >> >>
> >>
> >>
>
>

Mime
View raw message