Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@harmony.apache.org
Received-SPF: pass (athena.apache.org: domain of alexei.zakharov@gmail.com
 designates 209.85.146.180 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:to:subject:in-reply-to:mime-version
         :content-type:content-transfer-encoding:content-disposition
         :references;
        b=w3oGNGH7Okqxs+lsNxnEZDzAQEw8WRxgNayORtA6nonkQB+0H9NBNwWDUqr0XLX/Hl
         3Gz6IysU0H0VIt3gsmrAQmzvv4t1YJ0i9dayboeIGiro7JmOUG32YJuXQT9dK3PQXhGA
         oE9UyryhPfMUzEN7YPFVeykEhKjNN0a4sof3k=
Message-ID: <2c9597b90806110359u6705bb87u66631ecb552c76f5@mail.gmail.com>
Date: Wed, 11 Jun 2008 14:59:07 +0400
From: "Alexei Zakharov" <alexei.zakharov@gmail.com>
To: dev@harmony.apache.org
Subject: Re: [classlib][test] Migration to testNG?
In-Reply-To: <484FA217.50403@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <94d710af0806052021y490e9e12of6cea992c2495a83@mail.gmail.com>
	 <3b3f27c60806052031y1b457eb1x66d249e2b59dc089@mail.gmail.com>
	 <4848D471.9060402@gmail.com>
	 <3b3f27c60806061910o48ffac4fmf5cdfb4c052cdab6@mail.gmail.com>
	 <484FA217.50403@gmail.com>

As far as I understand in spite of the fact there were no
TestNG-related discussions since 2006 the problem is still relevant.
There are big exclude lists in some classlib modules still, and many
tests are excluded only because of a couple of failing methods.
Frankly speaking I'm not familiar with new feature introduced in Junit
4.4. Are there any enhancements that can help to resolve this
exclude-whole-class-because-of-one-bad-method issue?

Thanks,
Alexei

2008/6/11, Regis <xu.regis@gmail.com>:
>
> Nathan Beyer wrote:
> > On Fri, Jun 6, 2008 at 1:08 AM, Regis <xu.regis@gmail.com> wrote:
> >
> >
> > > Hi,
> > >
> > > Matcher and Assumptions are great ideas! Thanks Nathan.
> > > They would be very helpful for our new test cases. But I notice that
> > > Junit 4.4 doesn't support group which is very important feature for
> > > both old tests and new tests. We can partition our test suite and run
> > > them separately. It's make our tests more flexible and configurable,
> > > and it's the main reason we discuss to migrate to TestNG long time ago.
> > >
> >
> >
> > Don't we partition our tests already? Isn't that what the 'api' and 'impl'
> > folders are about?
> >
> Yes, but it's not enough. We have discussed and created a wiki page[1] about
> how
> to configuration and group harmony tests. The page is a little old, but I
> think the problems
> it tried to resolve still exist now. The partitions are not only include
> 'api' and 'impl', but also
> include partition of different os, architecture, partition of broken tests
> and level of tests.
> folder structure or exclude files can't help in this complex situation, so
> we need some tools
> to help us to deal with this, i think TestNG is suitable. If JUnit 4.4 can
> do it, i will vote to JUnit,
> update to a new version is always easier than switch to a new tool after
> all.
>
> [1] http://wiki.apache.org/harmony/Testing_Convention
>
> Best Regards,
> Regis.
>
>
> >
> > -Nathan
> >
> >
> >
> > >
> > > Best Regards,
> > > Regis.
> > >
> > >
> > > Nathan Beyer wrote:
> > >
> > >
> > > > That discussion was a very long time ago. Is there still value in
> TestNG?
> > > > I'd prefer to move to JUnit 4.4. All of our current tests will
> continue to
> > > > work and new tests can be implemented using the latest conventions and
> > > > older
> > > > tests can be updated as we get to them. JUnit 4.4 is a far cry from
> 4.0.
> > > >
> > > > Here's the things I think would be create for our use and testing in
> > > > general
> > > > - Matchers and the 'assertThat' - much more readable code and readable
> > > > failure messages
> > > > - Assumptions and the 'assumeThat' - allows methods to add statements
> that
> > > > guarantee that preconditions for the test are correct; this allows
> tests
> > > > to
> > > > fail such that you know it's an environment issue and not an actual
> test
> > > > failure
> > > >
> > > > If you're not familiar with matchers, check out this quick tutorial -
> > > > http://code.google.com/p/hamcrest/wiki/Tutorial.
> > > >
> > > > -Nathan
> > > >
> > > >
> > > > On Thu, Jun 5, 2008 at 10:21 PM, Sean Qiu <sean.xx.qiu@gmail.com>
> wrote:
> > > >
> > > >  Hi, all.
> > > >
> > > > > We had discussed the migration to testNG before and got some
> conclusions
> > > > > for
> > > > > grouping[1]
> > > > > including how to deal with boot path test[2]. Am i missing
> something?
> > > > > Is it still in our schedule? I think it's valueable to Harmony.
> > > > > I volunteer to carry out this job if no one objects.  Any other
> > > > > volunteers?
> > > > >
> > > > > IMHO, we can only add some ant tasks to integrate testng at the
> > > > > beginning.
> > > > > So our original junit tests can still work at the mean time when
> > > > > migrating.
> > > > > When one module's migration task is finished, we can judge the
> result
> > > > > to dertermine whether we should go on for other modules.
> > > > >
> > > > > Maybe we can create a branch for luni to start this work, shall we?
> > > > > therefore there won't be any impact on other's development.
> > > > > Once it is completed in the branch, we could merge it back to our
> trunk.
> > > > > Does it make sense?
> > > > >
> > > > > Any sugestions or comments are welcomed. Thanks very much.
> > > > >
> > > > > [1]
> http://wiki.apache.org/harmony/Testing_Convention
> > > > > [2]
> > > > >
> > > > >
> http://www.mail-archive.com/harmony-dev@incubator.apache.org/msg12413.html
> > > > > [3]
> http://testng.org/doc/documentation-main.html#annotations
> > > > >  --
> > > > > Best Regards
> > > > > Sean, Xiao Xia Qiu
> > > > >  China Software Development Lab, IBM
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> >
>