harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stepan Mishura" <stepan.mish...@gmail.com>
Subject Re: [classlib][instrument]Using Support_Exec framework(was Re: [classlib][Instrument] Documents for non-unit-tests)
Date Wed, 16 Aug 2006 11:47:55 GMT
On 8/16/06, Anton Luht wrote:
>
> IMHO analysis of exception stacktrace output is a bad idea. Tests
> should not produce uncatched exceptions unless they fail. Test authors
> should explicitly catch an exception and print some text to stdout.


 See [1] for testing details.

Reasons are:
>
> - exception messages are not standard, different JVMs can produce
> different output
> - localized messages can be printed, the test may fail because of locales
> - if test author expects IOException but subclass of this exception is
> thrown and printed, the test will fail while if the test was written
> in the way "catch IOException; print "PASSED" the test would pass


For example, how you will test that VM is aborted during start-up with
NoClassDefFoundError (not with IllegalArgumentException or any other
exception) if it is impossible to load requested class?

Thanks,
Stepan.

[1]
http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200608.mbox/%3c44D6AC2E.4000603@gmail.com%3e

On 8/16/06, Stepan Mishura <stepan.mishura@gmail.com> wrote:
> > On 8/16/06, Anton Luht wrote:
> > >
> > > Cool idea - rethrow exceptions from exec'ed java :) You exec jvm that
> > > dies because some exception (say, NPE) not catched, support_exec
> > > analyzes output, understands that NPE was thrown and rethrows it :)
> > > People who will look at those tests will certainly go mad.
> >
> >
> > You misunderstoond the idea for using support_exec - no exceptions
> > re-throwing should be made. A test will verify only that VM fails in
> > expected way (i.e. with expected error message).
> >
> > Do you have a better idea how to test that?
> >
> > Thanks,
> > Stepan.
> >
> > On 8/16/06, Jimmy, Jing Lv <firepure@gmail.com> wrote:
> > > > Paulex Yang wrote:
> > > > > Jimmy, Jing Lv wrote:
> > > > >> Hi,
> > > > >>
> > > > >>     I'm trying to write some unit tests of instrument. As
> discussed,
> > > > >> I'm using exec.
> > > > >>     Currently I've tried Support_Exec and find it fairly easy
to
> > > > >> use(Thanks Stepan :) ).
> > > > >>
> > > > >>     For resources (jar files and utility classes), I make a new
> > > > >> directory:
> > > > >>
> > >
> $Harmony/modules/instrument/src/test/resource/org/apache/harmony/tests/instrument
> > > > >> and put them there.
> > > > >>
> > > > >>     And I meet a problem here. Currently the default classpath
in
> > > > >> running test is "$Harmony/modules/instrument/bin", so it meets
> > > problem
> > > > >> in loading those utility classes (it can load jar files though).
> > > > >>     I remember Mark has plan to change the classpath in running
> > > > >> testcase, it'll be great if he make some progress.
> > > > >>
> > > > >>     An another problem is checking, currently the result is
> checked
> > > by
> > > > >> searching and compare in string(e.g, find "someException" in
> result).
> > > > >> But the framework seems fails if any exception is thrown out.
Can
> it
> > > > >> make a little change that it passes and returns the error message
> in
> > > > >> exec?
> > > > > Didn't catch up here, did you mean that you need to check the
> > > exception
> > > > > message as well as exception type?
> > > >
> > > > Not necessary to check messages. However we can not catch those
> > > > exceptions so we have to analysis output. For an example:
> > > >
> > > > run "java -javaagent:test.jar myclass"
> > > >
> > > > so get output:
> > > > Exception in thread "main"
> java.lang.reflect.InvocationTargetException
> > > >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> > > >         at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown
> Source)
> > > >         at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> > > Source)
> > > >         at java.lang.reflect.Method.invoke(Unknown Source)
> > > >         at
> > > > sun.instrument.InstrumentationImpl.loadClassAndCallPremain(Unknown
> So
> > > > urce)
> > > > Caused by: java.lang.IllegalAccessError
> > > > ...
> > > >
> > > > In this case we can search for "InvocationTargetException" in the
> output
> > > > to ensure the expected exception. Sometimes we may also check "cause
> by"
> > > > exceptions.
> > > >
> > > > This approach seems not perfect. Any comments/suggestions? Thanks!
> > > >
> > > > <snip>
> > > > --
> > > >
> > > > Best Regards!
> > > >
> > > > Jimmy, Jing Lv
> > > > China Software Development Lab, IBM
>
> ------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message