harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Ellison <t.p.elli...@gmail.com>
Subject Re: Unit test code in HARMONY-16
Date Mon, 16 Jan 2006 12:22:06 GMT
at the risk of getting dragged into this ;-)

I think George's objection is to the extension mechanism being used
(i.e. a custom subclass rather than a standard JUnit decoration [1]).

Adding more and more subclasses is not a practical way to go if you
think of all the different ways you may choose to extend the framework's
behaviour.

Regards,
Tim

[1] http://junit.sourceforge.net/javadoc/junit/extensions/TestDecorator.html

Mikhail Loenko wrote:
> Sorry, but I cannot catch what is the problem.
> If some output is annoying please send the test name and what is wrong with it.
> PerformanceTest class does not add any output to what the tests print.
> 
> For now the only implication from the fact that all the tests extend
> PerfromanceTest
> is that the security classes are well performance-tuned and on the majority of
> scenarios outperform "standard" Java implementations.
> 
> Thanks,
> Mikhail Loenko
> Intel Middleware Products Division
> 
> On 1/16/06, Geir Magnusson Jr <geir@pobox.com> wrote:
> 
>>
>>Loenko, Mikhail Y wrote:
>>
>>>Hi George
>>>
>>>We will remove all the tests and these QA guys will never disturb us :)
>>>
>>>Every time we remove a test we leave something untested.
>>>
>>>For example, SerializationTest is a base for all the tests that check
>>>serialization compatibility, and if we remove it because we do not
>>>think about serialization right now we will lose that compatibility.
>>>
>>>The same for performance, it is of importance still. Existing test suite
>>>allows us seeking performance regressions. Otherwise we will have
>>>to keep in sync two parallel test suites - one PerformanceTest based
>>>just for testing the performance and another one - for unit testing.
>>
>>I love this subject - unit tests an performance tests are different, as
>>can be regression tests, although one might argue that many of the
>>so-called regression tests are just unit tests that you forgot to do in
>>the first place.
>>
>>
>>>So, what is the noise? GUI is thinking that those base classes are
>>>tests?
>>>Maybe it makes sense to rename PerformanceTest to e.g. PerformanceTost
>>>and GUI will be happy?
>>
>>No. :/
>>
>>Can we change the output to be meaningful?  I remember a while back on
>>another project that I knew the timings of things on my machine just
>>from practice, and I could tell when we changed something that impacted
>>performance, because the tests ran longer.  This helped catch problems
>>early.
>>
>>geir
>>
>>
>>
>>>Thanks,
>>>Mikhail Loenko
>>>Intel Middleware Products Division
>>>
>>>
>>>>-----Original Message-----
>>>>From: George Harley1 [mailto:GHARLEY@uk.ibm.com]
>>>>Sent: Friday, January 13, 2006 7:24 PM
>>>>To: harmony-dev@incubator.apache.org
>>>>Subject: RE: Unit test code in HARMONY-16
>>>>
>>>>Hi Mikhail,
>>>>
>>>>Thanks for your very complete answer.
>>>>
>>>>
>>>>>At some point we had different functionality in the
>>>>>PerformanceTest but it seems to have died now. That is basically it.
>>>>
>>>>Do you see this class (and its SecurityTest and SerializationTest
>>>>subclasses) as candidates for removal then ? When I run the security
>>>
>>>unit
>>>
>>>>tests inside my IDE they add some extra lines to the console output
>>>
>>>but,
>>>
>>>>since I am not thinking about performance right now, that is just
>>>>"background noise".
>>>>
>>>>Perhaps additional performance-related functionality would be better
>>>
>>>moved
>>>
>>>>out of the test class' hierarchy and into some decorator class ? That
>>>
>>>way
>>>
>>>>would give developers a bit more flexibility running the tests with or
>>>>without the intervention of the performance measurement code. Sound
>>>>reasonable ?
>>>>
>>>>Best regards,
>>>>George
>>>>________________________________________
>>>>George C. Harley
>>>>
>>>>
>>>>
>>>>
>>>>"Loenko, Mikhail Y" <mikhail.y.loenko@intel.com>
>>>>13/01/2006 12:12
>>>>Please respond to
>>>>harmony-dev@incubator.apache.org
>>>>
>>>>
>>>>To
>>>><harmony-dev@incubator.apache.org>
>>>>cc
>>>>
>>>>Subject
>>>>RE: Unit test code in HARMONY-16
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>As far as we did not have special performance tests, we used unit tests
>>>>to measure performance, i.e. to compare performance of our classes to
>>>>performance of "standard" classes. So we ran in cycle a single unit
>>>
>>>test
>>>
>>>>on both when there are our security classes in bootclasspath and when
>>>>there are not. And compared time. (Of course, not all the tests passed
>>>>on "RI")
>>>>
>>>>Some unit tests print various logs that make execution time volatile.
>>>
>>>To
>>>
>>>>make it more stable we used log() instead of System.out.print() and in
>>>>the "performance mode" did not print anything. log() is defined in the
>>>>PerformanceTest. At some point we had different functionality in the
>>>>PerformanceTest but it seems to have died now. That is basically it.
>>>>
>>>>The results helped us to find a number of performance leaks and improve
>>>>overall quality of the code.
>>>>
>>>>Thanks,
>>>>Mikhail Loenko
>>>>Intel Middleware Products Division
>>>>
>>>>
>>>>
>>>>>-----Original Message-----
>>>>>From: George Harley1 [mailto:GHARLEY@uk.ibm.com]
>>>>>Sent: Thursday, January 12, 2006 10:05 PM
>>>>>To: harmony-dev@incubator.apache.org
>>>>>Subject: Unit test code in HARMONY-16
>>>>>
>>>>>Hi,
>>>>>
>>>>>I have been looking into running the JUnit tests included in
>>>
>>>HARMONY-16
>>>
>>>>in
>>>>
>>>>>my private sandbox. From what I have seen so far most (all ?) of the
>>>>
>>>>test
>>>>
>>>>>cases inherit from a base class PerformanceTest in the
>>>>>com.openintel.drl.security.test package. What is the purpose of this
>>>>
>>>>base
>>>>
>>>>>class ?
>>>>>
>>>>>Best regards,
>>>>>George
>>>>>________________________________________
>>>>>George C. Harley
>>>
>>>
> 

-- 

Tim Ellison (t.p.ellison@gmail.com)
IBM Java technology centre, UK.

Mime
View raw message