harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexei Zakharov" <alexei.zakha...@gmail.com>
Subject Re: [classlib][testing] excluding the failed tests
Date Mon, 10 Jul 2006 14:51:42 GMT
Thanks George & Tim, I was out during last week and  today was reading
threads from oldest to the newest. :)
I agree, general solution using TestSuites or even TestNG is better
than my temporary one. However, defining a general approach can take a
long period of time.  Anyway, let's move our discussion to that
thread.

2006/7/10, George Harley <george.c.harley@googlemail.com>:
> Alexei Zakharov wrote:
> > Hi,
> >
> >> If there are really useful tests that are being unnecessarily excluded
> >> by being in the same *Test class, then you may want to consider moving
> >> the failing tests into SecureRandom3Test and excluding that -- but by
> >> the sound of it all SecureRandom tests will be failing.
> >
> > I think it's a nice idea to do this at least for java.beans since
> > there are hundreds of useful workable tests excluded. After quite a
> > long time working with this module I have a strong wish to clean up
> > the mess.
> >
> > But probably we should define some naming pattern for class to put
> > excluded tests into. For example for XMLEncoderTest.java we can have
> > XMLEncoderTest_Disabled.java or XMLEncoderTest_Failed.java. In this
> > case we don't need to put extra "exclude" clause in the build.xml
> > since such name doesn't match **/*Test.java pattern (current). Another
> > variant is something like FAILED_XMLEncoderTest.java - matches the
> > pattern and needs the clause. Thoughts?
>
> Hi Alexei,
>
> Have you seen the discussion thread related to configuring our tests
> using suites [1] ? If not, then it seems to me that there is potential
> there for a simpler/quicker way of excluding or including tests without
> recourse to creating new files or renaming existing ones. What do you
> think ?
>
> Best regards,
> George
>
> [1]
> http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200607.mbox/%3c44ABB451.30806@googlemail.com%3e
>
>
> >
> >
> > 2006/7/6, Tim Ellison <t.p.ellison@gmail.com>:
> >> Vladimir Ivanov wrote:
> >> > More details: it is
> >> > org/apache/harmony/security/tests/java/security/SecureRandom2Test.java
> >> > test.
> >> > At present time it has 2 failing tests with messages about SHA1PRNG
> >> > algorithm (no support for SHA1PRNG provider).
> >> > Looks like it is valid tests for non implemented functionality,
> >> but, I'm
> >> > not
> >> > sure what to do with such TestCase(s): comment these 2 tests or
> >> move them
> >> > into separate TestCase.
> >> > Ideas?
> >>
> >> I'd prefer that we only use one mechanism for excluding tests, and today
> >> that is the excludes clause in the ant script.  So I suggest that you do
> >> option (4) below.
> >>
> >> If there are really useful tests that are being unnecessarily excluded
> >> by being in the same *Test class, then you may want to consider moving
> >> the failing tests into SecureRandom3Test and excluding that -- but by
> >> the sound of it all SecureRandom tests will be failing.
> >>
> >> > By the way, probably, it worth reviewing *all* excluded TestCases and:
> >> > 1.      Unexclude if all tests pass.
> >> > 2.      Report bug and provide patch for test to make it passing if it
> >> > failed due to bug in test.
> >> > 3.      Report bug (and provide patch) for implementation to make
> >> tests
> >> > passing, if it was/is bug in implementation and no such issue in JIRA.
> >> > 4.      Specify reasons for excluding TestCases in exclude list to
> >> make
> >> > further clean-up process easier.
> >> > 5.      Review results of this exclude list clean-up activity and then
> >> > decide what to do with the rest failing tests.
> >> >
> >> > I can do it starting next week. Do you think it worth doing?
> >> > Thanks, Vladimir
> >>
> >> Sounds great, thanks Vladimir.
> >>
> >> Regards,
> >> Tim
> >
> >
>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>


-- 
Alexei Zakharov,
Intel Middleware Product Division

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message