harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stepan Mishura" <stepan.mish...@gmail.com>
Subject Re: [classlib][Instrument] Documents for non-unit-tests (was Re:[classlib] Plan on work on java.lang.instrument)
Date Tue, 08 Aug 2006 10:10:56 GMT
On 8/8/06, Jimmy, Jing Lv wrote:
>
> Stepan Mishura wrote:
> > On 8/7/06, *Jimmy, Jing Lv*  wrote:
> >
> >     <SNIP>
> >     Sounds pretty good :)
> >     "exec" do helps, it can check simple situations.
> >     What I'm concern is that if the return code is not enough for some
> >     situations, e.g, what exception is thrown exactly, or what cause VM
> exit
> >     abnormally. IMHO, it is still necessary for us.
> >
> >
> > Hi Jimmy,
> >
> > See attached test. It demonstrates VM forking. The test saves VM
> > output and prints it in the end.
> > So I expect that unit tests for 'instrument' module can do the same:
> > fork VM with predefined args, save output and compare it with expected
> > output.
> >
> > BWT, we have utility class[1] in 'support' for this task. It makes sense
> > to improve it for our needs.
> >
> > Thanks,
> > Stepan.
> >
>
> Thanks Stepan, this is very attractive to me :)
>
> According to your approach, I believe now it can be automatized well, at
> least for many test.
>
> One concern from me is that, how do you handle output comparison? For an
> example, if VM halt with a exception, it output some message, so you get
> the output, find the exception name in it? And what can we do if it
> output some information only, which RI and Harmony may not appear the
> same?


Is the output format specified?

We may look for exact message or just find exception name in it. Also if
there are Harmony impl. specific tests we can place them into Harmony impl.
test suite.

However this is a good idea, if no objections, I shall follow this
> approach in writing some test of instrument.


As I said we need to improve Support_Exec support class. Currently it is
designed to fork only two selected VMs. At least we have to add Harmony VM.
And ideally it'd great to have support class that is able to fork any
compatible VM.

Thanks,
Stepan.

> [1]
> >
> http://svn.apache.org/viewvc/incubator/harmony/enhanced/classlib/trunk/support/src/test/java/tests/support/Support_Exec.java?revision=386058&view=markup
> > <
> http://svn.apache.org/viewvc/incubator/harmony/enhanced/classlib/trunk/support/src/test/java/tests/support/Support_Exec.java?revision=386058&view=markup
> >
> >
> >      > On 8/7/06, Jimmy, Jing Lv <firepure@gmail.com
> >     <mailto:firepure@gmail.com>> wrote:
> >      >>
> >      >> Geir Magnusson Jr wrote:
> >      >> >
> >      >> > Jimmy, Jing Lv wrote:
> >      >> >> Richard Liang wrote:
> >      >> >>>
> >      >> >>> Jimmy, Jing Lv wrote:
> >      >> >>>> Hi,
> >      >> >>>>
> >      >> >>>>     As it is hard to write unit test for package
> >     instrument, I now
> >      >> >>>> have an idea: write down some documents for details
of
> >     non-unit test
> >      >> >>>> cases for instrument. The detail of such test cases
> contain:
> >      >> >>> Do you mean it's hard to *write* unit test or it's hard
to
> >     integrate
> >      >> >>> the tests with Harmony build system?
> >      >> >>>
> >      >> >> Hard to write unit test, so record them in some document
may
> be a
> >      >> better
> >      >> >> way :)
> >      >> >
> >      >> > Why is it hard to create unit tests?
> >      >> >
> >      >>
> >      >> To test java.lang.instrument, mostly we need to start up a new
> >     VM to see
> >      >> its behavior. Its function(premain, etc) may be run before most
> >     classes
> >      >> loaded, and almost all exceptions will cause VM abort. And the
> >     result is
> >      >> hard to be checked automatic. What's more, extra java/c code is
> >     needed
> >      >> for the test.
> >      >>
> >      >> One thought was that we can use exec("java commands") in
> >     junit(is that
> >      >> OK to run exec in tests?), but how to check its output is still
> a
> >      >> problem.
> >      >>
> >      >> I'll put unit test if any, but I believe most necessary and
> >     important
> >      >> tests are non-unit tests.
> >      >>
> >      >> >>>> 1. The test run on which platform, including special
> >     environment;
> >      >> >>>> 2. The test description;
> >      >> >>>> 3. How to run test (command line, etc.);
> >      >> >>>> 4. test result;
> >      >> >>>> 5. resource (or links) for tests, including c/java
code for
> >     test if
> >      >> any.
> >      >> >>>>
> >      >> >>>>     I hope ensure the code quality in this way. Like
unit
> >     test, such
> >      >> >>>> tests can be checked and re-run for regression, though
> >      >> un-automatically.
> >      >> >>>>
> >      >> >>>>     And I'll put them in Harmony-wiki[1], and any
> >      >> test-description is
> >      >> >>>> welcome!. (Geir, I've found an advancement of wiki
:) )
> >      >> >>>>
> >      >> >>>>     Any suggestions?  :)
> >      >> >>>>
> >      >> >>>> [1]http://wiki.apache.org/harmony/INSTRUMENT
> >      >> >>>>
> >      >> >>>>> Geir Magnusson Jr wrote:
> >      >> >>>>> Never appeal to me using a Wiki as an authority
:)  It's
> like
> >      >> finding
> >      >> >>>>> something written in chalk on the sidewalk.
> >      >> >>>>>
> >      >> >>>>> However, if you had pointed out
> >      >> >>>>>
> >      >>
> >      >>>>>    http://incubator.apache.org/harmony/auth_cont_quest.html
> >     <http://incubator.apache.org/harmony/auth_cont_quest.html>
> >      >> >>>>>
> >      >> >>>>> I'd have agreed w/o having the opportunity to
make fun of
> >     the Wiki.
> >      >> >>>>>
> >      >> >>>>> So I agree :)
> >      >> >>>>>
> >      >> >>>>> geir
>
>


-- 
Thanks,
Stepan Mishura
Intel Middleware Products Division

------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message