Mailing-List: contact avro-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: avro-dev@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns;
	h=message-id:date:from:user-agent:mime-version:to:subject:
	references:in-reply-to:content-type:content-transfer-encoding;
	b=EEIrVD+IUSuwo+p7s5eM4zI48E2OJDSrmOi4YXLhNUb9RxuV7TUo3EIkt5TrR3A3
Message-ID: <49FB4365.2020001@yahoo-inc.com>
Date: Fri, 01 May 2009 11:45:57 -0700
From: Konstantin Boudnik <cos@yahoo-inc.com>
User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302)
MIME-Version: 1.0
To: "avro-dev@hadoop.apache.org" <avro-dev@hadoop.apache.org>
Subject: Re: TestNG in Avro [Was: Code reorg]
References: 
 <2C52DBBEC4855C438BB330CB0D3B46590131C90C@SNV-EXVS01.ds.corp.yahoo.com>
 <49FA8281.2040002@yahoo-inc.com>
In-Reply-To: <49FA8281.2040002@yahoo-inc.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Ok, I have completed the conversion of the unit tests for project Avro 
to be executed by TestNG framework. List of tests is built dynamically 
and all.

Here's the log of the execution:
--------------- start of log's snippet ---------------
test-java-ng:
    [testng] [TestNGAntTask] TESTNG PASSED @/tmp/testng34807 WHICH CONTAINS:
    [testng] [TestNGAntTask] -d
    [testng] [TestNGAntTask] /Users/cos/work/avro/trunk/build/test-output
    [testng] [TestNGAntTask] -sourcedir
    [testng] [TestNGAntTask] /Users/cos/work/avro/trunk/src/test/java
    [testng] [TestNGAntTask] -testclass
    [testng] [TestNGAntTask] 
/Users/cos/work/avro/trunk/build/test/classes/org/apache/avro/TestDataFile.class
    [testng] [TestNGAntTask] 
/Users/cos/work/avro/trunk/build/test/classes/org/apache/avro/TestFsData.class
    [testng] [TestNGAntTask] 
/Users/cos/work/avro/trunk/build/test/classes/org/apache/avro/TestProtocolGeneric.class
    [testng] [TestNGAntTask] 
/Users/cos/work/avro/trunk/build/test/classes/org/apache/avro/TestProtocolReflect.class
    [testng] [TestNGAntTask] 
/Users/cos/work/avro/trunk/build/test/classes/org/apache/avro/TestProtocolSpecific.class
    [testng] [TestNGAntTask] 
/Users/cos/work/avro/trunk/build/test/classes/org/apache/avro/TestReflect.class
    [testng] [TestNGAntTask] 
/Users/cos/work/avro/trunk/build/test/classes/org/apache/avro/TestSchema.class
    [testng] [TestNGAntTask] 
/Users/cos/work/avro/trunk/build/test/classes/org/apache/avro/io/TestValueReader.class
    [testng] [TestNGAntTask] -suitename
    [testng] [TestNGAntTask] AvroTestNG
    [testng] [TestNGAntTask] -testname
    [testng] [TestNGAntTask] Ant test
    [testng] [Parser] Running:
    [testng]   AvroTestNG
    [testng]
    [testng] 523 [main] INFO org.apache.avro.ipc.SocketServer - starting 
on 0.0.0.0/0.0.0.0:0
    [testng] 523 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO 
org.apache.avro.ipc.SocketServer - listening on /0.0.0.0
    [testng] 531 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO 
org.apache.avro.ipc.SocketServer - connection from /192.168.102.128:51751
    [testng] 531 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO 
org.apache.avro.ipc.SocketServer - listening on /0.0.0.0
    [testng] 763 [main] INFO org.apache.avro.ipc.SocketServer - starting 
on 0.0.0.0/0.0.0.0:0
    [testng] 764 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO 
org.apache.avro.ipc.SocketServer - listening on /0.0.0.0
    [testng] 764 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO 
org.apache.avro.ipc.SocketServer - connection from /192.168.102.128:51753
    [testng] 764 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO 
org.apache.avro.ipc.SocketServer - listening on /0.0.0.0
    [testng] 883 [main] INFO org.apache.avro.ipc.SocketServer - starting 
on 0.0.0.0/0.0.0.0:0
    [testng] 883 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO 
org.apache.avro.ipc.SocketServer - listening on /0.0.0.0
    [testng] 883 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO 
org.apache.avro.ipc.SocketServer - connection from /192.168.102.128:51755
    [testng] 884 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO 
org.apache.avro.ipc.SocketServer - listening on /0.0.0.0
    [testng] 888 [Connection for 
java.nio.channels.SocketChannel[connected local=/192.168.102.128:51754 
remote=/192.168.102.128:51755]] INFO org.apache.avro.TestProtocolGeneric 
- echo: {name=foo}
    [testng] 898 [Connection for 
java.nio.channels.SocketChannel[connected local=/192.168.102.128:51754 
remote=/192.168.102.128:51755]] INFO org.apache.avro.TestProtocolGeneric 
- echoBytes: java.nio.HeapByteBuffer[pos=0 lim=13717 cap=13717]
    [testng] 909 [Connection for 
java.nio.channels.SocketChannel[connected local=/192.168.102.128:51754 
remote=/192.168.102.128:51755]] INFO org.apache.avro.TestProtocolGeneric 
- hello: bob
    [testng] 1159 [main] INFO org.apache.avro.ipc.SocketServer - 
starting on 0.0.0.0/0.0.0.0:0
    [testng] 1160 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO 
org.apache.avro.ipc.SocketServer - listening on /0.0.0.0
    [testng] 1162 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO 
org.apache.avro.ipc.SocketServer - connection from /192.168.102.128:51757
    [testng] 1162 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO 
org.apache.avro.ipc.SocketServer - listening on /0.0.0.0
    [testng]
    [testng] ===============================================
    [testng] AvroTestNG
    [testng] Total tests run: 42, Failures: 0, Skips: 0
    [testng] ===============================================
    [testng]

BUILD SUCCESSFUL
--------------- end of log's snippet ---------------

A couple of lessons learned from the exercise:
   - JUnitConvertor isn't much help. I've found quite a number of test 
methods simply ignored by the application
   - a major chunk of conversion have to be done manually, e.g. sequence 
of the test method execution is needed to be written before hand, etc.
   - if I knew the product (Avro) and its testing better it would take 
much less time, because some of the issues weren't require any 
investigation for they become obvious after some initial acquittance 
with the code base structure, etc.


I'm creating new JIRA for this improvement and you should see the patch 
in a couple of hours.

If anyone has a better idea of how to submit it - I'd appreciate to hear 
an opinion from the development community.

Thanks,
   Cos

Konstantin Boudnik wrote:
> Sharad, Nigel.
> 
> I'm trying to finish the conversion of the junit tests into testng and 
> have accomplished a certain success. A couple of very special issues I 
> came across:
> 
>    - a generated interface build/test/src/org/apache/avro/Test.java was 
> a source of errors when TestNG imports we added into test files. I had 
> to rename test.js to generated.js and refactor few tests to work with 
> the new name. Generally, using names like Test for helper classes might 
> be a source of troubles.
> 
>    - it seems that some of the tests have to be executed in a certain 
> order. I probably should be able to derive the order of the execution by 
>   re-engineering build.xml file itself. However, before I'll jump on it 
> I'd like to know if someone here has any information about required java 
> test execution sequence.
> 
> So, looks like the conversion process goes Ok, but it isn't seamless and 
> require a significant amount of manual work.
> 
> Thanks for any information,
>   Cos
> 
> Sharad Agarwal wrote:
>> Sounds good. I think you can create a Jira and provide a patch.
>>>    - create an XML listing of tests to be executed (simple selection of
>>> **/*.class doesn't work in this case)
>> I don't know much about Test NG, but can we workaround this issue?
>> Maintaining an XML file seems to be an overhead.
>>
>> - Sharad