harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anton Luht" <anton.l...@gmail.com>
Subject Re: [classlib][instrument]Using Support_Exec framework(was Re: [classlib][Instrument] Documents for non-unit-tests)
Date Wed, 16 Aug 2006 11:22:06 GMT
IMHO analysis of exception stacktrace output is a bad idea. Tests
should not produce uncatched exceptions unless they fail. Test authors
should explicitly catch an exception and print some text to stdout.
Reasons are:

- exception messages are not standard, different JVMs can produce
different output
- localized messages can be printed, the test may fail because of locales
- if test author expects IOException but subclass of this exception is
thrown and printed, the test will fail while if the test was written
in the way "catch IOException; print "PASSED" the test would pass


On 8/16/06, Stepan Mishura <stepan.mishura@gmail.com> wrote:
> On 8/16/06, Anton Luht wrote:
> >
> > Cool idea - rethrow exceptions from exec'ed java :) You exec jvm that
> > dies because some exception (say, NPE) not catched, support_exec
> > analyzes output, understands that NPE was thrown and rethrows it :)
> > People who will look at those tests will certainly go mad.
>
>
> You misunderstoond the idea for using support_exec - no exceptions
> re-throwing should be made. A test will verify only that VM fails in
> expected way (i.e. with expected error message).
>
> Do you have a better idea how to test that?
>
> Thanks,
> Stepan.
>
> On 8/16/06, Jimmy, Jing Lv <firepure@gmail.com> wrote:
> > > Paulex Yang wrote:
> > > > Jimmy, Jing Lv wrote:
> > > >> Hi,
> > > >>
> > > >>     I'm trying to write some unit tests of instrument. As discussed,
> > > >> I'm using exec.
> > > >>     Currently I've tried Support_Exec and find it fairly easy to
> > > >> use(Thanks Stepan :) ).
> > > >>
> > > >>     For resources (jar files and utility classes), I make a new
> > > >> directory:
> > > >>
> > $Harmony/modules/instrument/src/test/resource/org/apache/harmony/tests/instrument
> > > >> and put them there.
> > > >>
> > > >>     And I meet a problem here. Currently the default classpath in
> > > >> running test is "$Harmony/modules/instrument/bin", so it meets
> > problem
> > > >> in loading those utility classes (it can load jar files though).
> > > >>     I remember Mark has plan to change the classpath in running
> > > >> testcase, it'll be great if he make some progress.
> > > >>
> > > >>     An another problem is checking, currently the result is checked
> > by
> > > >> searching and compare in string(e.g, find "someException" in result).
> > > >> But the framework seems fails if any exception is thrown out. Can
it
> > > >> make a little change that it passes and returns the error message
in
> > > >> exec?
> > > > Didn't catch up here, did you mean that you need to check the
> > exception
> > > > message as well as exception type?
> > >
> > > Not necessary to check messages. However we can not catch those
> > > exceptions so we have to analysis output. For an example:
> > >
> > > run "java -javaagent:test.jar myclass"
> > >
> > > so get output:
> > > Exception in thread "main" java.lang.reflect.InvocationTargetException
> > >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >         at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> > >         at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> > Source)
> > >         at java.lang.reflect.Method.invoke(Unknown Source)
> > >         at
> > > sun.instrument.InstrumentationImpl.loadClassAndCallPremain(Unknown So
> > > urce)
> > > Caused by: java.lang.IllegalAccessError
> > > ...
> > >
> > > In this case we can search for "InvocationTargetException" in the output
> > > to ensure the expected exception. Sometimes we may also check "cause by"
> > > exceptions.
> > >
> > > This approach seems not perfect. Any comments/suggestions? Thanks!
> > >
> > > <snip>
> > > --
> > >
> > > Best Regards!
> > >
> > > Jimmy, Jing Lv
> > > China Software Development Lab, IBM
> > >
> >
> >
>
>
> --
> Thanks,
> Stepan Mishura
> Intel Middleware Products Division
>
> ------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>


-- 
Regards,
Anton Luht,
Intel Middleware Products Division

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message