harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Geir Magnusson Jr <g...@pobox.com>
Subject Re: Unit test code in HARMONY-16
Date Mon, 16 Jan 2006 11:22:06 GMT


Loenko, Mikhail Y wrote:
> Hi George
> 
> We will remove all the tests and these QA guys will never disturb us :)
> 
> Every time we remove a test we leave something untested.
> 
> For example, SerializationTest is a base for all the tests that check 
> serialization compatibility, and if we remove it because we do not 
> think about serialization right now we will lose that compatibility.
> 
> The same for performance, it is of importance still. Existing test suite
> allows us seeking performance regressions. Otherwise we will have
> to keep in sync two parallel test suites - one PerformanceTest based
> just for testing the performance and another one - for unit testing.

I love this subject - unit tests an performance tests are different, as 
can be regression tests, although one might argue that many of the 
so-called regression tests are just unit tests that you forgot to do in 
the first place.

> 
> So, what is the noise? GUI is thinking that those base classes are
> tests?
> Maybe it makes sense to rename PerformanceTest to e.g. PerformanceTost
> and GUI will be happy?

No. :/

Can we change the output to be meaningful?  I remember a while back on 
another project that I knew the timings of things on my machine just 
from practice, and I could tell when we changed something that impacted 
performance, because the tests ran longer.  This helped catch problems 
early.

geir


> 
> Thanks,
> Mikhail Loenko
> Intel Middleware Products Division
> 
>> -----Original Message-----
>> From: George Harley1 [mailto:GHARLEY@uk.ibm.com]
>> Sent: Friday, January 13, 2006 7:24 PM
>> To: harmony-dev@incubator.apache.org
>> Subject: RE: Unit test code in HARMONY-16
>>
>> Hi Mikhail,
>>
>> Thanks for your very complete answer.
>>
>>> At some point we had different functionality in the
>>> PerformanceTest but it seems to have died now. That is basically it.
>> Do you see this class (and its SecurityTest and SerializationTest
>> subclasses) as candidates for removal then ? When I run the security
> unit
>> tests inside my IDE they add some extra lines to the console output
> but,
>> since I am not thinking about performance right now, that is just
>> "background noise".
>>
>> Perhaps additional performance-related functionality would be better
> moved
>> out of the test class' hierarchy and into some decorator class ? That
> way
>> would give developers a bit more flexibility running the tests with or
>> without the intervention of the performance measurement code. Sound
>> reasonable ?
>>
>> Best regards,
>> George
>> ________________________________________
>> George C. Harley
>>
>>
>>
>>
>> "Loenko, Mikhail Y" <mikhail.y.loenko@intel.com>
>> 13/01/2006 12:12
>> Please respond to
>> harmony-dev@incubator.apache.org
>>
>>
>> To
>> <harmony-dev@incubator.apache.org>
>> cc
>>
>> Subject
>> RE: Unit test code in HARMONY-16
>>
>>
>>
>>
>>
>>
>> As far as we did not have special performance tests, we used unit tests
>> to measure performance, i.e. to compare performance of our classes to
>> performance of "standard" classes. So we ran in cycle a single unit
> test
>> on both when there are our security classes in bootclasspath and when
>> there are not. And compared time. (Of course, not all the tests passed
>> on "RI")
>>
>> Some unit tests print various logs that make execution time volatile.
> To
>> make it more stable we used log() instead of System.out.print() and in
>> the "performance mode" did not print anything. log() is defined in the
>> PerformanceTest. At some point we had different functionality in the
>> PerformanceTest but it seems to have died now. That is basically it.
>>
>> The results helped us to find a number of performance leaks and improve
>> overall quality of the code.
>>
>> Thanks,
>> Mikhail Loenko
>> Intel Middleware Products Division
>>
>>
>>> -----Original Message-----
>>> From: George Harley1 [mailto:GHARLEY@uk.ibm.com]
>>> Sent: Thursday, January 12, 2006 10:05 PM
>>> To: harmony-dev@incubator.apache.org
>>> Subject: Unit test code in HARMONY-16
>>>
>>> Hi,
>>>
>>> I have been looking into running the JUnit tests included in
> HARMONY-16
>> in
>>> my private sandbox. From what I have seen so far most (all ?) of the
>> test
>>> cases inherit from a base class PerformanceTest in the
>>> com.openintel.drl.security.test package. What is the purpose of this
>> base
>>> class ?
>>>
>>> Best regards,
>>> George
>>> ________________________________________
>>> George C. Harley
> 
> 

Mime
View raw message