harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From George Harley1 <GHAR...@uk.ibm.com>
Subject Re: [jira] Commented: (HARMONY-31) Move peformance timing of unit tests into a decorator class.
Date Wed, 18 Jan 2006 13:44:02 GMT
Hi Mikhail, 

> The messages are important to analyze failures also.

What is JUnit's own failure reporting mechanism not providing to you ? 


> And the possibility to test perfromance is also important

Yes it is. But - to return to the original point of the HARMONY-31 JIRA 
issue - this can be done without the need to either bring in a superclass 
in the test hierarchy and/or scatter logging calls around the test code. 
Performance testing using JUnit should be done in a transparent manner 
using a decorator. As an example of what I mean please take a look at the 
JUnitPerf site at http://clarkware.com/software/JUnitPerf.html . JUnitPerf 
is an extension to JUnit that helps with the creation of performance tests 
that match existing unit tests. It does this using a decorator design. 
This provides for a separation of concerns that benefits developers and 
performance engineers. 

The decorator approach means that we effectively *extend* JUnit with 
simple wrapper classes in which we can make full use of the JUnit API to 
give us the additional behaviour needed when running the tests (maybe this 
could be used to give whatever extra failure analysis data you say is 
lacking ?). And if somebody doesn't want the extension behaviour ? They 
just run the suite without the custom wrappers. 


> When we have a powerful performance suite we may revisit this.

Don't you mean when we have a powerful unit test suite ? 

Take care, 
George
________________________________________
George C. Harley





Mikhail Loenko <mloenko@gmail.com> 
18/01/2006 12:57
Please respond to
harmony-dev@incubator.apache.org


To
harmony-dev@incubator.apache.org
cc

Subject
Re: [jira] Commented: (HARMONY-31) Move peformance timing of unit tests 
into a decorator class.






Well, those messages were important for developers when
they were writing code and tests. Then tests came to repository 'as is'.

The messages are important to analyze failures also.
And the possibility to test perfromance is also important

For me any option that does not break functionality in favor of beauty 
looks
good. I suggest swithcing off logging by default and ether keep
PerformanceTest
super class as is or replace it with a simple Logger class.

When we have a powerful performance suite we may revisit this.

Thanks,
Mikhail.

On 1/18/06, Tim Ellison <t.p.ellison@gmail.com> wrote:
> Absolutely right -- writing meaningful performance tests is hard.
> Implementing your own Logger would not solve the problem though<g>.
>
> Best to avoid the 'This test worked OK' log messages altogether, and
> stick to assertions.
>
> Regards,
> Tim
>
> Mikhail Loenko wrote:
> > It might be a problem...
> >
> > When we use java.util.logging we do not just compare performance of 
security
> > API functions, the result is also depends on difference in performance 
of
> > java.util.logging in standard classes vs. Harmony classes. So if we 
use
> > non-trivial functionality from there then our results will be spoiled 
a little.
> >
> > Will investigate more...
> >
> > Thanks,
> > Mikhail.
> >
> > On 1/17/06, Tim Ellison <t.p.ellison@gmail.com> wrote:
> >> neither is the Logger class -- so my point is if you are going to 
write
> >> some logging code why not do it in java.util.logging?  You may choose 
to
> >> only do simple stubs for now until somebody steps up to do a real 
impl.
> >>
> >> Regards,
> >> Tim
> >>
> >> Mikhail Loenko wrote:
> >>> It's not yet implemented.
> >>>
> >>> thanks,
> >>> Mikhail
> >>>
> >>> On 1/17/06, Tim Ellison <t.p.ellison@gmail.com> wrote:
> >>>> Why not use java.util.logging?
> >>>>
> >>>> Regards,
> >>>> Tim
> >>>>
> >>>> Mikhail Loenko (JIRA) wrote:
> >>>>>     [ 
http://issues.apache.org/jira/browse/HARMONY-31?page=comments#action_12362910 
]
> >>>>>
> >>>>> Mikhail Loenko commented on HARMONY-31:
> >>>>> ---------------------------------------
> >>>>>
> >>>>> This is not what I meant.
> >>>>>
> >>>>> I was going to create a Logger class at this point like this:
> >>>>>
> >>>>> public class Logger {
> >>>>>         public static boolean printAllowed = false;
> >>>>>       public static void log(String message) {
> >>>>>               if (printAllowed) System.out.print(message);
> >>>>>       }
> >>>>>       public static void logln(String message) {
> >>>>>               if (printAllowed) System.out.println(message);
> >>>>>       }
> >>>>>       public static void logError(String message) {
> >>>>>               if (printAllowed) System.err.print(message);
> >>>>>       }
> >>>>>       public static void loglnError(String message) {
> >>>>>               if (printAllowed) System.err.println(message);
> >>>>>       }
> >>>>> }
> >>>>>
> >>>>> And replace log() with Logger.log() everywhere in the tests.
> >>>>>
> >>>>> All the remaining functionality in the PerformanceTest is 
obsolete.
> >>>>>
> >>>>>
> >>>>>> Move peformance timing of unit tests into a decorator class.
> >>>>>> ------------------------------------------------------------
> >>>>>>
> >>>>>>          Key: HARMONY-31
> >>>>>>          URL: http://issues.apache.org/jira/browse/HARMONY-31
> >>>>>>      Project: Harmony
> >>>>>>         Type: Improvement
> >>>>>>     Reporter: George Harley
> >>>>>>     Assignee: Geir Magnusson Jr
> >>>>>>     Priority: Minor
> >>>>>>  Attachments: PerfDecorator.java
> >>>>>>
> >>>>>> There has been some low-level discussion on the dev mailing
list 
recently about the inclusion of performance-related logging code near the 
top of a unit test class inheritance hierarchy (see 
com.openintel.drl.security.test.PerformanceTest in the HARMONY-16 
contribution). This particular issue suggests an alternative way of adding 
in timing code but without making it the responsibility of the unit tests 
themselves and without the need to introduce a class in the inheritance 
hierarchy.
> >>>>>> The basic approach is to exploit the 
junit.extensions.TestDecorator type in the JUnit API to add in timing 
behaviour before and after each test method runs. This will be 
demonstrated with some simple sample code.
> >>>> --
> >>>>
> >>>> Tim Ellison (t.p.ellison@gmail.com)
> >>>> IBM Java technology centre, UK.
> >>>>
> >> --
> >>
> >> Tim Ellison (t.p.ellison@gmail.com)
> >> IBM Java technology centre, UK.
> >>
> >
>
> --
>
> Tim Ellison (t.p.ellison@gmail.com)
> IBM Java technology centre, UK.
>



Mime
View raw message