harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Geir Magnusson Jr." <g...@pobox.com>
Subject Re: [testing][classlib] x-lists clean-up
Date Mon, 19 Feb 2007 12:41:26 GMT

On Feb 19, 2007, at 5:53 AM, tatyana doubtsova wrote:

> The comment in auth build.xml
> <!-- excluded to avoid random failures -->
> is dated 07.09.06
> There were several clean-up commits to
> org/apache/harmony/auth/tests/module/Krb5LoginModuleTest.java   
> after this.
> All the tests pretend to be returned were run iteratively - PASSED
> All the modules with returned tests were run iteratively for 20  
> times in
> once mode - PASSED.
>
> In case of possible failures in iterative classlib tests runs under  
> cc, I'll
> put the test in corresponding x-list.
>
> I also agree that JIRA number or/and comments in x-lists would be very
> helpful.

+1

On thing also to keep in mind is that tests could fail due to side  
effects or VM issues that aren't clear at the time of exclusion.  It  
would be nice if there was a mechanical way to do this - for example,  
maybe we can add a flag that also runs the excluded tests so we can  
then examine the log and see what did pass that we didn't expect to.

This still feels like we're re-inventing the wheel.  TestNG  
supposedly has a lot of these features - has anyone tried looking at  
TestNG recently?  We talked about switching a while ago..

geir


>
> Thanks,
> Tanya
>
>
> On 2/19/07, Alexei Zakharov <alexei.zakharov@gmail.com> wrote:
>>
>> At the same time there are tests that reside in exclude lists by
>> absolutely unknown reason. I.e. initially they were added because of
>> some issue that was fixed a quarter ago. So IMHO the only way to get
>> rid of such unwanted excludes is to run *all* tests periodically and
>> check do they fail anymore.  This is an exhausting work - thanks
>> Tatyana for doing this. BTW I think it will be also nice if we put
>> JIRA number or any other useful comment directly to exclude lists  
>> just
>> as a reminder why this test is here.
>>
>> Thanks,
>>
>> 2007/2/19, Mikhail Loenko <mloenko@gmail.com>:
>> > It would be good to find what were the reasons for excluding  
>> those 12
>> tests.
>> >
>> > For example, the first test in the patch:
>> > org/apache/harmony/auth/tests/module/Krb5LoginModuleTest.java
>> >
>> > was originally excluded in build.xml with the following comment:
>> > <!-- The test is under development     -->
>> > <!-- excluded to avoid random failures -->
>> >
>> > Since that the code of the test was not modified, so the test is  
>> still
>> > incomplete and should not be returned back.
>> >
>> > Other tests may have similar problems too
>> >
>> > Thanks,
>> > Mikhail
>> >
>> > 2007/2/19, tatyana doubtsova <tatyanadoubtsova@gmail.com>:
>> > > Hi,
>> > > I've run excluded classlib tests on win ia32 drlvm|ibmvme,  
>> linux ia32
>> > > drlvm|ibmvme, linux em64t drlvm and found out that 12 tests from
>> common
>> > > x-lists passed on all 5 configurations. The tests are from  
>> auth, luni,
>> > > nio_char, rmi, sound, text, lang-manegement, and security  
>> modules.
>> I've
>> > > prepared the patch for x-lists clean-up:
>> > > https://issues.apache.org/jira/browse/HARMONY-3194.   
>> Commiters, would
>> you,
>> > > please, have a look and apply, if there are no objections?
>> > >
>> > > Thanks,
>> > > Tanya
>>
>>
>> --
>> Alexei Zakharov,
>> Intel ESSD
>>


Mime
View raw message