Mailing-List: contact harmony-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: harmony-dev@incubator.apache.org
Received-SPF: pass (asf.osuosl.org: domain of mloenko@gmail.com designates
 66.249.92.200 as permitted sender)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
        s=beta; d=gmail.com;
        h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references;
        b=uJHGitdRvQCUmoGUw49jlolQizRkRgzIzT4aoQai1xx0OYu8m9vArRqUZh+0nU4167DJSrNvWW/gpSardUKsfX1afLuxCposVbAlcihd/rSrW2QopFVyNe27TqCbNAF8gT756pRlAxhWtwDEg0iWVKjUp1pCZm9lJt3HEYicqy4=
Message-ID: <906dd82e0601160404t39b7cc0bx599344af71a3280d@mail.gmail.com>
Date: Mon, 16 Jan 2006 18:04:55 +0600
From: Mikhail Loenko <mloenko@gmail.com>
To: harmony-dev@incubator.apache.org
Subject: Re: Unit test code in HARMONY-16
In-Reply-To: <43CB81DE.4020203@pobox.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
References: <6694B22B6436BC43B429958787E45498011D7A78@mssmsx402nb>
	 <43CB81DE.4020203@pobox.com>

Sorry, but I cannot catch what is the problem.
If some output is annoying please send the test name and what is wrong with=
 it.
PerformanceTest class does not add any output to what the tests print.

For now the only implication from the fact that all the tests extend
PerfromanceTest
is that the security classes are well performance-tuned and on the majority=
 of
scenarios outperform "standard" Java implementations.

Thanks,
Mikhail Loenko
Intel Middleware Products Division

On 1/16/06, Geir Magnusson Jr <geir@pobox.com> wrote:
>
>
> Loenko, Mikhail Y wrote:
> > Hi George
> >
> > We will remove all the tests and these QA guys will never disturb us :)
> >
> > Every time we remove a test we leave something untested.
> >
> > For example, SerializationTest is a base for all the tests that check
> > serialization compatibility, and if we remove it because we do not
> > think about serialization right now we will lose that compatibility.
> >
> > The same for performance, it is of importance still. Existing test suit=
e
> > allows us seeking performance regressions. Otherwise we will have
> > to keep in sync two parallel test suites - one PerformanceTest based
> > just for testing the performance and another one - for unit testing.
>
> I love this subject - unit tests an performance tests are different, as
> can be regression tests, although one might argue that many of the
> so-called regression tests are just unit tests that you forgot to do in
> the first place.
>
> >
> > So, what is the noise? GUI is thinking that those base classes are
> > tests?
> > Maybe it makes sense to rename PerformanceTest to e.g. PerformanceTost
> > and GUI will be happy?
>
> No. :/
>
> Can we change the output to be meaningful?  I remember a while back on
> another project that I knew the timings of things on my machine just
> from practice, and I could tell when we changed something that impacted
> performance, because the tests ran longer.  This helped catch problems
> early.
>
> geir
>
>
> >
> > Thanks,
> > Mikhail Loenko
> > Intel Middleware Products Division
> >
> >> -----Original Message-----
> >> From: George Harley1 [mailto:GHARLEY@uk.ibm.com]
> >> Sent: Friday, January 13, 2006 7:24 PM
> >> To: harmony-dev@incubator.apache.org
> >> Subject: RE: Unit test code in HARMONY-16
> >>
> >> Hi Mikhail,
> >>
> >> Thanks for your very complete answer.
> >>
> >>> At some point we had different functionality in the
> >>> PerformanceTest but it seems to have died now. That is basically it.
> >> Do you see this class (and its SecurityTest and SerializationTest
> >> subclasses) as candidates for removal then ? When I run the security
> > unit
> >> tests inside my IDE they add some extra lines to the console output
> > but,
> >> since I am not thinking about performance right now, that is just
> >> "background noise".
> >>
> >> Perhaps additional performance-related functionality would be better
> > moved
> >> out of the test class' hierarchy and into some decorator class ? That
> > way
> >> would give developers a bit more flexibility running the tests with or
> >> without the intervention of the performance measurement code. Sound
> >> reasonable ?
> >>
> >> Best regards,
> >> George
> >> ________________________________________
> >> George C. Harley
> >>
> >>
> >>
> >>
> >> "Loenko, Mikhail Y" <mikhail.y.loenko@intel.com>
> >> 13/01/2006 12:12
> >> Please respond to
> >> harmony-dev@incubator.apache.org
> >>
> >>
> >> To
> >> <harmony-dev@incubator.apache.org>
> >> cc
> >>
> >> Subject
> >> RE: Unit test code in HARMONY-16
> >>
> >>
> >>
> >>
> >>
> >>
> >> As far as we did not have special performance tests, we used unit test=
s
> >> to measure performance, i.e. to compare performance of our classes to
> >> performance of "standard" classes. So we ran in cycle a single unit
> > test
> >> on both when there are our security classes in bootclasspath and when
> >> there are not. And compared time. (Of course, not all the tests passed
> >> on "RI")
> >>
> >> Some unit tests print various logs that make execution time volatile.
> > To
> >> make it more stable we used log() instead of System.out.print() and in
> >> the "performance mode" did not print anything. log() is defined in the
> >> PerformanceTest. At some point we had different functionality in the
> >> PerformanceTest but it seems to have died now. That is basically it.
> >>
> >> The results helped us to find a number of performance leaks and improv=
e
> >> overall quality of the code.
> >>
> >> Thanks,
> >> Mikhail Loenko
> >> Intel Middleware Products Division
> >>
> >>
> >>> -----Original Message-----
> >>> From: George Harley1 [mailto:GHARLEY@uk.ibm.com]
> >>> Sent: Thursday, January 12, 2006 10:05 PM
> >>> To: harmony-dev@incubator.apache.org
> >>> Subject: Unit test code in HARMONY-16
> >>>
> >>> Hi,
> >>>
> >>> I have been looking into running the JUnit tests included in
> > HARMONY-16
> >> in
> >>> my private sandbox. From what I have seen so far most (all ?) of the
> >> test
> >>> cases inherit from a base class PerformanceTest in the
> >>> com.openintel.drl.security.test package. What is the purpose of this
> >> base
> >>> class ?
> >>>
> >>> Best regards,
> >>> George
> >>> ________________________________________
> >>> George C. Harley
> >
> >
>