Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@harmony.apache.org
Received-SPF: pass (athena.apache.org: domain of xu.regis@gmail.com designates
 209.85.146.183 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:user-agent:mime-version:to:subject:references
         :in-reply-to:content-type:content-transfer-encoding;
        b=K7qMAfPEXLJ16zIHQQYC4/mE8Wa+f3ql2v47qI1bVwTLA3xsd7ISGFsRD532clpG6c
         szrfBpdjTDWIDH5B+INFzFsnP11RCmpqC1syOUdlsQ0IyAbLobxCTIpfi0jgwxae/rfM
         aaJkJjAuN4JfJYbxVpBdpyNfjfCbH4ErVKpVc=
Message-ID: <484FBB29.7020407@gmail.com>
Date: Wed, 11 Jun 2008 19:46:49 +0800
From: Regis <xu.regis@gmail.com>
User-Agent: Thunderbird 2.0.0.14 (Windows/20080421)
MIME-Version: 1.0
To: dev@harmony.apache.org
Subject: Re: [classlib][test] Migration to testNG?
References: <94d710af0806052021y490e9e12of6cea992c2495a83@mail.gmail.com>
	 <3b3f27c60806052031y1b457eb1x66d249e2b59dc089@mail.gmail.com>
	 <4848D471.9060402@gmail.com>
	 <3b3f27c60806061910o48ffac4fmf5cdfb4c052cdab6@mail.gmail.com>
	 <484FA217.50403@gmail.com>
 <2c9597b90806110359u6705bb87u66631ecb552c76f5@mail.gmail.com>
In-Reply-To: <2c9597b90806110359u6705bb87u66631ecb552c76f5@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit


Junit 4 has @ignore annotation which of target is method, which can 
ignore failing methods.
But some modules, such as luni, have different exclude lists on 
different os or vm, I haven't
found any ways to do this using @ignore annotation. While TestNG's 
@group annotation
can do this easily, just define groups for different exclude lists.

Best Regards,
Regis.

Alexei Zakharov wrote:
> As far as I understand in spite of the fact there were no
> TestNG-related discussions since 2006 the problem is still relevant.
> There are big exclude lists in some classlib modules still, and many
> tests are excluded only because of a couple of failing methods.
> Frankly speaking I'm not familiar with new feature introduced in Junit
> 4.4. Are there any enhancements that can help to resolve this
> exclude-whole-class-because-of-one-bad-method issue?
> 
> Thanks,
> Alexei
> 
> 2008/6/11, Regis <xu.regis@gmail.com>:
>> Nathan Beyer wrote:
>>> On Fri, Jun 6, 2008 at 1:08 AM, Regis <xu.regis@gmail.com> wrote:
>>>
>>>
>>>> Hi,
>>>>
>>>> Matcher and Assumptions are great ideas! Thanks Nathan.
>>>> They would be very helpful for our new test cases. But I notice that
>>>> Junit 4.4 doesn't support group which is very important feature for
>>>> both old tests and new tests. We can partition our test suite and run
>>>> them separately. It's make our tests more flexible and configurable,
>>>> and it's the main reason we discuss to migrate to TestNG long time ago.
>>>>
>>>
>>> Don't we partition our tests already? Isn't that what the 'api' and 'impl'
>>> folders are about?
>>>
>> Yes, but it's not enough. We have discussed and created a wiki page[1] about
>> how
>> to configuration and group harmony tests. The page is a little old, but I
>> think the problems
>> it tried to resolve still exist now. The partitions are not only include
>> 'api' and 'impl', but also
>> include partition of different os, architecture, partition of broken tests
>> and level of tests.
>> folder structure or exclude files can't help in this complex situation, so
>> we need some tools
>> to help us to deal with this, i think TestNG is suitable. If JUnit 4.4 can
>> do it, i will vote to JUnit,
>> update to a new version is always easier than switch to a new tool after
>> all.
>>
>> [1] http://wiki.apache.org/harmony/Testing_Convention
>>
>> Best Regards,
>> Regis.
>>
>>
>>> -Nathan
>>>
>>>
>>>
>>>> Best Regards,
>>>> Regis.
>>>>
>>>>
>>>> Nathan Beyer wrote:
>>>>
>>>>
>>>>> That discussion was a very long time ago. Is there still value in
>> TestNG?
>>>>> I'd prefer to move to JUnit 4.4. All of our current tests will
>> continue to
>>>>> work and new tests can be implemented using the latest conventions and
>>>>> older
>>>>> tests can be updated as we get to them. JUnit 4.4 is a far cry from
>> 4.0.
>>>>> Here's the things I think would be create for our use and testing in
>>>>> general
>>>>> - Matchers and the 'assertThat' - much more readable code and readable
>>>>> failure messages
>>>>> - Assumptions and the 'assumeThat' - allows methods to add statements
>> that
>>>>> guarantee that preconditions for the test are correct; this allows
>> tests
>>>>> to
>>>>> fail such that you know it's an environment issue and not an actual
>> test
>>>>> failure
>>>>>
>>>>> If you're not familiar with matchers, check out this quick tutorial -
>>>>> http://code.google.com/p/hamcrest/wiki/Tutorial.
>>>>>
>>>>> -Nathan
>>>>>
>>>>>
>>>>> On Thu, Jun 5, 2008 at 10:21 PM, Sean Qiu <sean.xx.qiu@gmail.com>
>> wrote:
>>>>>  Hi, all.
>>>>>
>>>>>> We had discussed the migration to testNG before and got some
>> conclusions
>>>>>> for
>>>>>> grouping[1]
>>>>>> including how to deal with boot path test[2]. Am i missing
>> something?
>>>>>> Is it still in our schedule? I think it's valueable to Harmony.
>>>>>> I volunteer to carry out this job if no one objects.  Any other
>>>>>> volunteers?
>>>>>>
>>>>>> IMHO, we can only add some ant tasks to integrate testng at the
>>>>>> beginning.
>>>>>> So our original junit tests can still work at the mean time when
>>>>>> migrating.
>>>>>> When one module's migration task is finished, we can judge the
>> result
>>>>>> to dertermine whether we should go on for other modules.
>>>>>>
>>>>>> Maybe we can create a branch for luni to start this work, shall we?
>>>>>> therefore there won't be any impact on other's development.
>>>>>> Once it is completed in the branch, we could merge it back to our
>> trunk.
>>>>>> Does it make sense?
>>>>>>
>>>>>> Any sugestions or comments are welcomed. Thanks very much.
>>>>>>
>>>>>> [1]
>> http://wiki.apache.org/harmony/Testing_Convention
>>>>>> [2]
>>>>>>
>>>>>>
>> http://www.mail-archive.com/harmony-dev@incubator.apache.org/msg12413.html
>>>>>> [3]
>> http://testng.org/doc/documentation-main.html#annotations
>>>>>>  --
>>>>>> Best Regards
>>>>>> Sean, Xiao Xia Qiu
>>>>>>  China Software Development Lab, IBM
>>>>>>
>>>>>>
>>>>>>
>>>
>