harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Qiu" <sean.xx....@gmail.com>
Subject Re: [classlib][test] Migration to testNG?
Date Thu, 12 Jun 2008 05:54:39 GMT
Group failed test by annotation doesn't mean we want to keep the issues hidden.
On the contrary, we want to keep track of them.
We can run all failed test to verify if it is fixed.

2008/6/12 Nathan Beyer <nbeyer@gmail.com>:
> With annotation-based tests, just remove the annotation and that method is
> no longer treated as a test. However, the same thing could be done today by
> just changing the name of the method to NOT start with 'test'. I don't see
> any approach being good though as it keeps the issues hidden; nothing makes
> them stand out.
>
> -Nathan
>
> On Wed, Jun 11, 2008 at 5:59 AM, Alexei Zakharov <alexei.zakharov@gmail.com>
> wrote:
>
>> As far as I understand in spite of the fact there were no
>> TestNG-related discussions since 2006 the problem is still relevant.
>> There are big exclude lists in some classlib modules still, and many
>> tests are excluded only because of a couple of failing methods.
>> Frankly speaking I'm not familiar with new feature introduced in Junit
>> 4.4. Are there any enhancements that can help to resolve this
>> exclude-whole-class-because-of-one-bad-method issue?
>>
>> Thanks,
>> Alexei
>>
>> 2008/6/11, Regis <xu.regis@gmail.com>:
>> >
>> > Nathan Beyer wrote:
>> > > On Fri, Jun 6, 2008 at 1:08 AM, Regis <xu.regis@gmail.com> wrote:
>> > >
>> > >
>> > > > Hi,
>> > > >
>> > > > Matcher and Assumptions are great ideas! Thanks Nathan.
>> > > > They would be very helpful for our new test cases. But I notice that
>> > > > Junit 4.4 doesn't support group which is very important feature for
>> > > > both old tests and new tests. We can partition our test suite and
run
>> > > > them separately. It's make our tests more flexible and configurable,
>> > > > and it's the main reason we discuss to migrate to TestNG long time
>> ago.
>> > > >
>> > >
>> > >
>> > > Don't we partition our tests already? Isn't that what the 'api' and
>> 'impl'
>> > > folders are about?
>> > >
>> > Yes, but it's not enough. We have discussed and created a wiki page[1]
>> about
>> > how
>> > to configuration and group harmony tests. The page is a little old, but I
>> > think the problems
>> > it tried to resolve still exist now. The partitions are not only include
>> > 'api' and 'impl', but also
>> > include partition of different os, architecture, partition of broken
>> tests
>> > and level of tests.
>> > folder structure or exclude files can't help in this complex situation,
>> so
>> > we need some tools
>> > to help us to deal with this, i think TestNG is suitable. If JUnit 4.4
>> can
>> > do it, i will vote to JUnit,
>> > update to a new version is always easier than switch to a new tool after
>> > all.
>> >
>> > [1] http://wiki.apache.org/harmony/Testing_Convention
>> >
>> > Best Regards,
>> > Regis.
>> >
>> >
>> > >
>> > > -Nathan
>> > >
>> > >
>> > >
>> > > >
>> > > > Best Regards,
>> > > > Regis.
>> > > >
>> > > >
>> > > > Nathan Beyer wrote:
>> > > >
>> > > >
>> > > > > That discussion was a very long time ago. Is there still value
in
>> > TestNG?
>> > > > > I'd prefer to move to JUnit 4.4. All of our current tests will
>> > continue to
>> > > > > work and new tests can be implemented using the latest conventions
>> and
>> > > > > older
>> > > > > tests can be updated as we get to them. JUnit 4.4 is a far cry
from
>> > 4.0.
>> > > > >
>> > > > > Here's the things I think would be create for our use and testing
>> in
>> > > > > general
>> > > > > - Matchers and the 'assertThat' - much more readable code and
>> readable
>> > > > > failure messages
>> > > > > - Assumptions and the 'assumeThat' - allows methods to add
>> statements
>> > that
>> > > > > guarantee that preconditions for the test are correct; this allows
>> > tests
>> > > > > to
>> > > > > fail such that you know it's an environment issue and not an
actual
>> > test
>> > > > > failure
>> > > > >
>> > > > > If you're not familiar with matchers, check out this quick tutorial
>> -
>> > > > > http://code.google.com/p/hamcrest/wiki/Tutorial.
>> > > > >
>> > > > > -Nathan
>> > > > >
>> > > > >
>> > > > > On Thu, Jun 5, 2008 at 10:21 PM, Sean Qiu <sean.xx.qiu@gmail.com>
>> > wrote:
>> > > > >
>> > > > >  Hi, all.
>> > > > >
>> > > > > > We had discussed the migration to testNG before and got
some
>> > conclusions
>> > > > > > for
>> > > > > > grouping[1]
>> > > > > > including how to deal with boot path test[2]. Am i missing
>> > something?
>> > > > > > Is it still in our schedule? I think it's valueable to Harmony.
>> > > > > > I volunteer to carry out this job if no one objects.  Any
other
>> > > > > > volunteers?
>> > > > > >
>> > > > > > IMHO, we can only add some ant tasks to integrate testng
at the
>> > > > > > beginning.
>> > > > > > So our original junit tests can still work at the mean time
when
>> > > > > > migrating.
>> > > > > > When one module's migration task is finished, we can judge
the
>> > result
>> > > > > > to dertermine whether we should go on for other modules.
>> > > > > >
>> > > > > > Maybe we can create a branch for luni to start this work,
shall
>> we?
>> > > > > > therefore there won't be any impact on other's development.
>> > > > > > Once it is completed in the branch, we could merge it back
to our
>> > trunk.
>> > > > > > Does it make sense?
>> > > > > >
>> > > > > > Any sugestions or comments are welcomed. Thanks very much.
>> > > > > >
>> > > > > > [1]
>> > http://wiki.apache.org/harmony/Testing_Convention
>> > > > > > [2]
>> > > > > >
>> > > > > >
>> >
>> http://www.mail-archive.com/harmony-dev@incubator.apache.org/msg12413.html
>> > > > > > [3]
>> > http://testng.org/doc/documentation-main.html#annotations
>> > > > > >  --
>> > > > > > Best Regards
>> > > > > > Sean, Xiao Xia Qiu
>> > > > > >  China Software Development Lab, IBM
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> > >
>> >
>>
>



-- 
Best Regards
Sean, Xiao Xia Qiu

China Software Development Lab, IBM

Mime
View raw message