harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikhail Loenko <mloe...@gmail.com>
Subject Re: Unit test code in HARMONY-16
Date Mon, 16 Jan 2006 12:36:15 GMT
OK, now I see. I'll provide a script to convert the tests.

Thanks,
Mikhail

On 1/16/06, Tim Ellison <t.p.ellison@gmail.com> wrote:
> at the risk of getting dragged into this ;-)
>
> I think George's objection is to the extension mechanism being used
> (i.e. a custom subclass rather than a standard JUnit decoration [1]).
>
> Adding more and more subclasses is not a practical way to go if you
> think of all the different ways you may choose to extend the framework's
> behaviour.
>
> Regards,
> Tim
>
> [1] http://junit.sourceforge.net/javadoc/junit/extensions/TestDecorator.html
>
> Mikhail Loenko wrote:
> > Sorry, but I cannot catch what is the problem.
> > If some output is annoying please send the test name and what is wrong with it.
> > PerformanceTest class does not add any output to what the tests print.
> >
> > For now the only implication from the fact that all the tests extend
> > PerfromanceTest
> > is that the security classes are well performance-tuned and on the majority of
> > scenarios outperform "standard" Java implementations.
> >
> > Thanks,
> > Mikhail Loenko
> > Intel Middleware Products Division
> >
> > On 1/16/06, Geir Magnusson Jr <geir@pobox.com> wrote:
> >
> >>
> >>Loenko, Mikhail Y wrote:
> >>
> >>>Hi George
> >>>
> >>>We will remove all the tests and these QA guys will never disturb us :)
> >>>
> >>>Every time we remove a test we leave something untested.
> >>>
> >>>For example, SerializationTest is a base for all the tests that check
> >>>serialization compatibility, and if we remove it because we do not
> >>>think about serialization right now we will lose that compatibility.
> >>>
> >>>The same for performance, it is of importance still. Existing test suite
> >>>allows us seeking performance regressions. Otherwise we will have
> >>>to keep in sync two parallel test suites - one PerformanceTest based
> >>>just for testing the performance and another one - for unit testing.
> >>
> >>I love this subject - unit tests an performance tests are different, as
> >>can be regression tests, although one might argue that many of the
> >>so-called regression tests are just unit tests that you forgot to do in
> >>the first place.
> >>
> >>
> >>>So, what is the noise? GUI is thinking that those base classes are
> >>>tests?
> >>>Maybe it makes sense to rename PerformanceTest to e.g. PerformanceTost
> >>>and GUI will be happy?
> >>
> >>No. :/
> >>
> >>Can we change the output to be meaningful?  I remember a while back on
> >>another project that I knew the timings of things on my machine just
> >>from practice, and I could tell when we changed something that impacted
> >>performance, because the tests ran longer.  This helped catch problems
> >>early.
> >>
> >>geir
> >>
> >>
> >>
> >>>Thanks,
> >>>Mikhail Loenko
> >>>Intel Middleware Products Division
> >>>
> >>>
> >>>>-----Original Message-----
> >>>>From: George Harley1 [mailto:GHARLEY@uk.ibm.com]
> >>>>Sent: Friday, January 13, 2006 7:24 PM
> >>>>To: harmony-dev@incubator.apache.org
> >>>>Subject: RE: Unit test code in HARMONY-16
> >>>>
> >>>>Hi Mikhail,
> >>>>
> >>>>Thanks for your very complete answer.
> >>>>
> >>>>
> >>>>>At some point we had different functionality in the
> >>>>>PerformanceTest but it seems to have died now. That is basically
it.
> >>>>
> >>>>Do you see this class (and its SecurityTest and SerializationTest
> >>>>subclasses) as candidates for removal then ? When I run the security
> >>>
> >>>unit
> >>>
> >>>>tests inside my IDE they add some extra lines to the console output
> >>>
> >>>but,
> >>>
> >>>>since I am not thinking about performance right now, that is just
> >>>>"background noise".
> >>>>
> >>>>Perhaps additional performance-related functionality would be better
> >>>
> >>>moved
> >>>
> >>>>out of the test class' hierarchy and into some decorator class ? That
> >>>
> >>>way
> >>>
> >>>>would give developers a bit more flexibility running the tests with or
> >>>>without the intervention of the performance measurement code. Sound
> >>>>reasonable ?
> >>>>
> >>>>Best regards,
> >>>>George
> >>>>________________________________________
> >>>>George C. Harley
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>"Loenko, Mikhail Y" <mikhail.y.loenko@intel.com>
> >>>>13/01/2006 12:12
> >>>>Please respond to
> >>>>harmony-dev@incubator.apache.org
> >>>>
> >>>>
> >>>>To
> >>>><harmony-dev@incubator.apache.org>
> >>>>cc
> >>>>
> >>>>Subject
> >>>>RE: Unit test code in HARMONY-16
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>As far as we did not have special performance tests, we used unit tests
> >>>>to measure performance, i.e. to compare performance of our classes to
> >>>>performance of "standard" classes. So we ran in cycle a single unit
> >>>
> >>>test
> >>>
> >>>>on both when there are our security classes in bootclasspath and when
> >>>>there are not. And compared time. (Of course, not all the tests passed
> >>>>on "RI")
> >>>>
> >>>>Some unit tests print various logs that make execution time volatile.
> >>>
> >>>To
> >>>
> >>>>make it more stable we used log() instead of System.out.print() and in
> >>>>the "performance mode" did not print anything. log() is defined in the
> >>>>PerformanceTest. At some point we had different functionality in the
> >>>>PerformanceTest but it seems to have died now. That is basically it.
> >>>>
> >>>>The results helped us to find a number of performance leaks and improve
> >>>>overall quality of the code.
> >>>>
> >>>>Thanks,
> >>>>Mikhail Loenko
> >>>>Intel Middleware Products Division
> >>>>
> >>>>
> >>>>
> >>>>>-----Original Message-----
> >>>>>From: George Harley1 [mailto:GHARLEY@uk.ibm.com]
> >>>>>Sent: Thursday, January 12, 2006 10:05 PM
> >>>>>To: harmony-dev@incubator.apache.org
> >>>>>Subject: Unit test code in HARMONY-16
> >>>>>
> >>>>>Hi,
> >>>>>
> >>>>>I have been looking into running the JUnit tests included in
> >>>
> >>>HARMONY-16
> >>>
> >>>>in
> >>>>
> >>>>>my private sandbox. From what I have seen so far most (all ?) of
the
> >>>>
> >>>>test
> >>>>
> >>>>>cases inherit from a base class PerformanceTest in the
> >>>>>com.openintel.drl.security.test package. What is the purpose of this
> >>>>
> >>>>base
> >>>>
> >>>>>class ?
> >>>>>
> >>>>>Best regards,
> >>>>>George
> >>>>>________________________________________
> >>>>>George C. Harley
> >>>
> >>>
> >
>
> --
>
> Tim Ellison (t.p.ellison@gmail.com)
> IBM Java technology centre, UK.
>

Mime
View raw message