harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Geir Magnusson Jr." <g...@pobox.com>
Subject Re: [classlib][build] exclude list impl issues
Date Mon, 29 Jan 2007 16:39:20 GMT

On Jan 29, 2007, at 8:45 AM, Vladimir Ivanov wrote:

> On 1/29/07, Geir Magnusson Jr. <geir@pobox.com> wrote:
>>
>>
>> On Jan 28, 2007, at 11:28 PM, Vladimir Ivanov wrote:
>>
>> > OK. Current changes for exclude lists were integrated and now we  
>> have
>> > 3-level exclude list:
>> >
>> > exlude.common - tests, that failed over all platforms
>> >
>> > exclude.<platform> - tests, that failed over specific platform only
>> >
>> > exclude.<platform>.interm - tests that failed time to time over
>> > specific
>> > platform only.
>>
>> Quick q - why separate interm out?  why not just put in the platform
>> file?
>
>
> To do exclude lists clean up process easier. To delete test from the
> platform list you should check that this test passed one time. To  
> delete
> test from 'intermittently failed' exclude list each test should be  
> run in
> cycle.

:/

Doesn't the first case simply mean you got lucky?

>
>>
>> > Any file in this chain may be skipped. Final exclude lists are
>> > generated at
>> > the build time and stored to the ${hy.hdk}/build directory. Thanks
>> > to Alexei
>> > Zakharov for this changes.
>>
>> Another quick q - why not just glom things together in memory?
>
>
>
> If we will run tests against HDK it will nice to skip excluded  
> tests. For
> this reason exclude lists should be stored somewhere in the built  
> product.
>

That makes perfect sense.  Thanks

geir

>>
>> >
>> >
>> > But I have one more question to discuss: should we use the tests  
>> from
>> > 'intermittently failed' exclude list for preintegration testing?
>>
>> What does that mean?
>
>
>
> OK. Now pre-integration test include 'ant test' with requirements  
> that all
> tests should be passed. While we have intermittently failed tests this
> target sometimes report 'failed' status for these tests without  
> correlation
> with commit changes. We have 2 options here:
>
> - exclude these tests in the platform-specific ex-list
>
> - exclude these tests in the special ex-list
>
> Each option has pro and contra: in the first case we can miss the  
> regression
> when intermittently failed test became always failed or delete test  
> from
> exclude lists if it passed only one time.
>
> In the second case we have some overhead to support special exclude  
> lists.
>
> Note to detect regression the 'default' mode should be 'on (run)
> intermittently failed tests' to test commit changes and 'off' to  
> run tests
> under CC.
>
> I'll add switches to on/off levels for resulting exclude lists.
>
> thanks, Vladimir
>
>
>
>> > If we use
>> > it we may miss some regression when intermittently failed test will
>> > failed
>> > constantly however if we does not use it we need to run test twice
>> > sometimes.
>> > What is correct behavior?
>> >
>> >  thanks, Vladimir
>> >
>> >
>> >
>> > On 1/28/07, Alex Blewitt <alex.blewitt@gmail.com> wrote:
>> >>
>> >> Yeah, +1 for using common exclude lists. It makes it easier when
>> >> Harmony gets ported to other operating systems. And I don't see  
>> the
>> >> benefit of having empty lists in that case; and if nothing's  
>> failing,
>> >> you don't need an empty list either :-)
>> >>
>> >> Alex.
>> >>
>> >> On 28/01/07, Alexey Petrenko <alexey.a.petrenko@gmail.com> wrote:
>> >> > +1 from me for using common exclude lists and removing empty  
>> lists.
>> >> >
>> >> > SY, Alexey
>> >> >
>> >> > 2007/1/16, Alexey Varlamov <alexey.v.varlamov@gmail.com>:
>> >> > > Folks,
>> >> > >
>> >> > > I've some concerns with recent updates for x-list support: can
>> >> we use
>> >> > > a common macros instead of copy-pasting 4 new targets into
>> >> build.xml
>> >> > > for each module?
>> >> > > Or, if we can neglect creating a compiled x-file
>> >> > > (${hy.hdk}/build/<module>.exclude), just use "if" attribute
of
>> >> > > <excludesfile>, like this:
>> >> > >
>> >> > > <available property="x.list.exist"
>> >> > > file="exclude.${hy.platform}.${hy.test.vm.name}"/>
>> >> > > ...
>> >> > >             <batchtest>
>> >> > >                 <fileset dir="${src.test.java}">
>> >> > >                     <include name="**/*Test.java"/>
>> >> > >                     <excludesfile name="exclude.common"/>
>> >> > >                     <excludesfile name="${exclude.file} if="
>> >> x.list.exist" />
>> >> > >                 </fileset>
>> >> > >             </batchtest>
>> >> > >
>> >> > > Also, I suggest to delete empty x-list remained after  
>> introducing
>> >> common lists.
>> >> > >
>> >> > > Another issue is with "hy.test.vm.name", it was a surprise for
>> >> me that
>> >> > > it is not autodetected yet. Most obvious way to get it is  
>> to read
>> >> > > "java.vm.name" property, this only requires running trivial  
>> test.
>> >> > >
>> >> > > --
>> >> > > Alexey
>> >> > >
>> >> >
>> >>
>>
>>


Mime
View raw message