harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jimmy, Jing Lv" <firep...@gmail.com>
Subject Re: [classlib][instrument]Using Support_Exec framework(was Re: [classlib][Instrument] Documents for non-unit-tests)
Date Thu, 17 Aug 2006 02:05:07 GMT
Anton Luht wrote:
> IMHO analysis of exception stacktrace output is a bad idea. Tests
> should not produce uncatched exceptions unless they fail. Test authors
> should explicitly catch an exception and print some text to stdout.
> Reasons are:
> 
> - exception messages are not standard, different JVMs can produce
> different output
> - localized messages can be printed, the test may fail because of locales
> - if test author expects IOException but subclass of this exception is
> thrown and printed, the test will fail while if the test was written
> in the way "catch IOException; print "PASSED" the test would pass
> 
> 

Ah, we are trying to catch the exact exception name, not its message. I 
agree parsing the message is a bad idea, but find an exact exception 
name in the output is still stable, am I right?

In fact, I have no idea to catch an exception in "exec", it surely 
contains no such function, right?

> On 8/16/06, Stepan Mishura <stepan.mishura@gmail.com> wrote:
>> On 8/16/06, Anton Luht wrote:
>> >
>> > Cool idea - rethrow exceptions from exec'ed java :) You exec jvm that
>> > dies because some exception (say, NPE) not catched, support_exec
>> > analyzes output, understands that NPE was thrown and rethrows it :)
>> > People who will look at those tests will certainly go mad.
>>
>>
>> You misunderstoond the idea for using support_exec - no exceptions
>> re-throwing should be made. A test will verify only that VM fails in
>> expected way (i.e. with expected error message).
>>
>> Do you have a better idea how to test that?
>>
>> Thanks,
>> Stepan.
>>
>> On 8/16/06, Jimmy, Jing Lv <firepure@gmail.com> wrote:
>> > > Paulex Yang wrote:
>> > > > Jimmy, Jing Lv wrote:
>> > > >> Hi,
>> > > >>
>> > > >>     I'm trying to write some unit tests of instrument. As 
>> discussed,
>> > > >> I'm using exec.
>> > > >>     Currently I've tried Support_Exec and find it fairly easy
to
>> > > >> use(Thanks Stepan :) ).
>> > > >>
>> > > >>     For resources (jar files and utility classes), I make a new
>> > > >> directory:
>> > > >>
>> > 
>> $Harmony/modules/instrument/src/test/resource/org/apache/harmony/tests/instrument

>>
>> > > >> and put them there.
>> > > >>
>> > > >>     And I meet a problem here. Currently the default classpath
in
>> > > >> running test is "$Harmony/modules/instrument/bin", so it meets
>> > problem
>> > > >> in loading those utility classes (it can load jar files though).
>> > > >>     I remember Mark has plan to change the classpath in running
>> > > >> testcase, it'll be great if he make some progress.
>> > > >>
>> > > >>     An another problem is checking, currently the result is 
>> checked
>> > by
>> > > >> searching and compare in string(e.g, find "someException" in 
>> result).
>> > > >> But the framework seems fails if any exception is thrown out.

>> Can it
>> > > >> make a little change that it passes and returns the error 
>> message in
>> > > >> exec?
>> > > > Didn't catch up here, did you mean that you need to check the
>> > exception
>> > > > message as well as exception type?
>> > >
>> > > Not necessary to check messages. However we can not catch those
>> > > exceptions so we have to analysis output. For an example:
>> > >
>> > > run "java -javaagent:test.jar myclass"
>> > >
>> > > so get output:
>> > > Exception in thread "main" 
>> java.lang.reflect.InvocationTargetException
>> > >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native 
>> Method)
>> > >         at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown 
>> Source)
>> > >         at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
>> > Source)
>> > >         at java.lang.reflect.Method.invoke(Unknown Source)
>> > >         at
>> > > sun.instrument.InstrumentationImpl.loadClassAndCallPremain(Unknown So
>> > > urce)
>> > > Caused by: java.lang.IllegalAccessError
>> > > ...
>> > >
>> > > In this case we can search for "InvocationTargetException" in the 
>> output
>> > > to ensure the expected exception. Sometimes we may also check 
>> "cause by"
>> > > exceptions.
>> > >
>> > > This approach seems not perfect. Any comments/suggestions? Thanks!
>> > >
>> > > <snip>
>> > > --
>> > >
>> > > Best Regards!
>> > >
>> > > Jimmy, Jing Lv
>> > > China Software Development Lab, IBM
>> > >
>> >
>> >
>>
>>
>> -- 
>> Thanks,
>> Stepan Mishura
>> Intel Middleware Products Division
>>
>> ------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
> 
> 


-- 

Best Regards!

Jimmy, Jing Lv
China Software Development Lab, IBM

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message