Mailing-List: contact harmony-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: harmony-dev@incubator.apache.org
Received-SPF: neutral (asf.osuosl.org: 202.81.18.154 is neither permitted nor
 denied by domain of firepure@gmail.com)
Message-ID: <44D8530F.1020601@gmail.com>
Date: Tue, 08 Aug 2006 17:02:07 +0800
From: "Jimmy, Jing Lv" <firepure@gmail.com>
User-Agent: Thunderbird 1.5.0.5 (Windows/20060719)
MIME-Version: 1.0
To: harmony-dev@incubator.apache.org
Subject: Re: [classlib][Instrument] Documents for non-unit-tests (was
 Re:[classlib]
 Plan on work on java.lang.instrument)
References: <44D01E9A.30800@gmail.com>
	 <6e47b64f0608012205t2275cf58q5c1e03d106033d99@mail.gmail.com>
	 <44D03412.4080506@pobox.com>
 <44D1AE24.8000202@gmail.com>	 <44D2DDD8.8060107@gmail.com>
 <44D696E6.2040103@gmail.com>	 <44D6A3CA.5040602@pobox.com>
 <44D6AC2E.4000603@gmail.com>
	 <e66844de0608070230q3b8efd29oeb76913b8aaaf10c@mail.gmail.com>
	 <44D70E1A.3090104@gmail.com>
 <6e47b64f0608072303h65d0bd24v2337297fe0af4bbc@mail.gmail.com>
In-Reply-To: <6e47b64f0608072303h65d0bd24v2337297fe0af4bbc@mail.gmail.com>
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

Stepan Mishura wrote:
> On 8/7/06, *Jimmy, Jing Lv*  wrote:
> 
>     <SNIP>
>     Sounds pretty good :)
>     "exec" do helps, it can check simple situations.
>     What I'm concern is that if the return code is not enough for some
>     situations, e.g, what exception is thrown exactly, or what cause VM exit
>     abnormally. IMHO, it is still necessary for us.
> 
>  
> Hi Jimmy,
>  
> See attached test. It demonstrates VM forking. The test saves VM 
> output and prints it in the end.
> So I expect that unit tests for 'instrument' module can do the same: 
> fork VM with predefined args, save output and compare it with expected 
> output.
>  
> BWT, we have utility class[1] in 'support' for this task. It makes sense 
> to improve it for our needs.
>  
> Thanks,
> Stepan.
>  

Thanks Stepan, this is very attractive to me :)

According to your approach, I believe now it can be automatized well, at 
least for many test.

One concern from me is that, how do you handle output comparison? For an 
example, if VM halt with a exception, it output some message, so you get 
the output, find the exception name in it? And what can we do if it 
output some information only, which RI and Harmony may not appear the same?

However this is a good idea, if no objections, I shall follow this 
approach in writing some test of instrument.

> [1] 
> http://svn.apache.org/viewvc/incubator/harmony/enhanced/classlib/trunk/support/src/test/java/tests/support/Support_Exec.java?revision=386058&view=markup 
> <http://svn.apache.org/viewvc/incubator/harmony/enhanced/classlib/trunk/support/src/test/java/tests/support/Support_Exec.java?revision=386058&view=markup>
> 
>      > On 8/7/06, Jimmy, Jing Lv <firepure@gmail.com
>     <mailto:firepure@gmail.com>> wrote:
>      >>
>      >> Geir Magnusson Jr wrote:
>      >> >
>      >> > Jimmy, Jing Lv wrote:
>      >> >> Richard Liang wrote:
>      >> >>>
>      >> >>> Jimmy, Jing Lv wrote:
>      >> >>>> Hi,
>      >> >>>>
>      >> >>>>     As it is hard to write unit test for package
>     instrument, I now
>      >> >>>> have an idea: write down some documents for details of
>     non-unit test
>      >> >>>> cases for instrument. The detail of such test cases contain:
>      >> >>> Do you mean it's hard to *write* unit test or it's hard to
>     integrate
>      >> >>> the tests with Harmony build system?
>      >> >>>
>      >> >> Hard to write unit test, so record them in some document may be a
>      >> better
>      >> >> way :)
>      >> >
>      >> > Why is it hard to create unit tests?
>      >> >
>      >>
>      >> To test java.lang.instrument, mostly we need to start up a new
>     VM to see
>      >> its behavior. Its function(premain, etc) may be run before most
>     classes
>      >> loaded, and almost all exceptions will cause VM abort. And the
>     result is
>      >> hard to be checked automatic. What's more, extra java/c code is
>     needed
>      >> for the test.
>      >>
>      >> One thought was that we can use exec("java commands") in
>     junit(is that
>      >> OK to run exec in tests?), but how to check its output is still a
>      >> problem.
>      >>
>      >> I'll put unit test if any, but I believe most necessary and
>     important
>      >> tests are non-unit tests.
>      >>
>      >> >>>> 1. The test run on which platform, including special
>     environment;
>      >> >>>> 2. The test description;
>      >> >>>> 3. How to run test (command line, etc.);
>      >> >>>> 4. test result;
>      >> >>>> 5. resource (or links) for tests, including c/java code for
>     test if
>      >> any.
>      >> >>>>
>      >> >>>>     I hope ensure the code quality in this way. Like unit
>     test, such
>      >> >>>> tests can be checked and re-run for regression, though
>      >> un-automatically.
>      >> >>>>
>      >> >>>>     And I'll put them in Harmony-wiki[1], and any
>      >> test-description is
>      >> >>>> welcome!. (Geir, I've found an advancement of wiki :) )
>      >> >>>>
>      >> >>>>     Any suggestions?  :)
>      >> >>>>
>      >> >>>> [1]http://wiki.apache.org/harmony/INSTRUMENT
>      >> >>>>
>      >> >>>>> Geir Magnusson Jr wrote:
>      >> >>>>> Never appeal to me using a Wiki as an authority :)  It's like
>      >> finding
>      >> >>>>> something written in chalk on the sidewalk.
>      >> >>>>>
>      >> >>>>> However, if you had pointed out
>      >> >>>>>
>      >>
>      >>>>>    http://incubator.apache.org/harmony/auth_cont_quest.html
>     <http://incubator.apache.org/harmony/auth_cont_quest.html>
>      >> >>>>>
>      >> >>>>> I'd have agreed w/o having the opportunity to make fun of
>     the Wiki.
>      >> >>>>>
>      >> >>>>> So I agree :)
>      >> >>>>>
>      >> >>>>> geir
> 
> 
> -- 
> Thanks,
> Stepan Mishura
> Intel Middleware Products Division
> 
> ------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org 
> <mailto:harmony-dev-unsubscribe@incubator.apache.org>
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org 
> <mailto:harmony-dev-help@incubator.apache.org>
> 
> 
> ------------------------------------------------------------------------
> 
> import java.io.*;
> 
> public class test {
> 
>     public static void main(String []av) throws Exception {
>         Runtime r = Runtime.getRuntime();
> 
>         String[] args = new String[3];
>         
>         args[0] = System.getProperty("java.home") + "/bin/java";
>         args[1] = "-showversion";
>         args[2] = "NoSuchClass";
>         
>         Process proc = r.exec(args);
> 
>         MyReader err = new MyReader(proc.getErrorStream());
>         MyReader out = new MyReader(proc.getInputStream());
>             
>         new Thread(err).start();
>         new Thread(out).start();
>                                 
>         int status = proc.waitFor();
> 
>         System.out.println("======OUT=========");
>         System.out.println(out.out.toString());
>         System.out.println("======ERR=========");
>         System.out.println(err.out.toString());
>         System.out.println("======EXIT=========");
>         System.out.println(status);
> 
>     }
> }
> 
> class MyReader implements Runnable{
> 
>     InputStream in;
>     ByteArrayOutputStream out = new ByteArrayOutputStream();
> 
>     public MyReader(InputStream in){
>         this.in = in;
>     }
>         
>     public void run() {
>         try {
>             for (int b = in.read(); b != -1; b = in.read()) {
>                 out.write(b);
>             }
>         } catch (IOException e) {
>             e.printStackTrace();
>         }
>     }
> }
> 
> 
> ------------------------------------------------------------------------
> 
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org


-- 

Best Regards!

Jimmy, Jing Lv
China Software Development Lab, IBM

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org