harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mikhail Loenko" <mloe...@gmail.com>
Subject Re: [classlib][math] Location of performance tests (was: Re: performance improvement for java.math package)
Date Thu, 22 Jun 2006 14:43:33 GMT
The first thing that came into my mind is as far as it is a regression test,
put somewhere next to the test both old and new Math implementations,
measure 'golden' performances and than measure where the current performance
in comparison to the golden is.

This scenario might be simplified. If your optimization works e.g. for numbers
less than say 1'000'000, you can compare performance for 999'999 and
1'000'001

Thanks,
Mikhail

2006/6/22, Vladimir Strigun <vstrigun@gmail.com>:
> Mikhail,
>
> I can convert it to JUnit, but I'm not pretty sure about returning
> pass/fail. When you think test should return fail?
> Results of test execution can be different on different VM's, it also
> dependent of machine speed, etc.
>
> Thanks,
> Vladimir.
>
> On 6/22/06, Mikhail Loenko <mloenko@gmail.com> wrote:
> > I've confused all the things. Sorry. Of course the tests should go to
> > math/src/test/performance
> >
> > Vladimir, is it possible to convert the test to JUnit format and make them
> > report pass/fail status rather than printing log?
> >
> > Thanks,
> > Mikhail
> >
> > 2006/6/21, Mikhail Loenko <mloenko@gmail.com>:
> > > I'd like to bring it to common judgment.
> > >
> > > AFAIU we have two basic options for performance tests location: make
> > > them module level or make a top-level tests subtree that would contain
> > > all types of the tests except for unit tests
> > >
> > > BTW, In the testing convention my intent was to cover unit tests only.
> > > Though we did not agree which exactly tests are "unit", so I avoided that word.
> > > But I definitely did not mean performance, stress and whatever special types
> > > of the tests. If no one objects I'll add some clarification to the conventions
> > > proposal.
> > >
> > > Thanks,
> > > Mikhail
> > >
> > > 2006/6/20, Vladimir Strigun <vstrigun@gmail.com>:
> > > > On 6/20/06, Mikhail Loenko <mloenko@gmail.com> wrote:
> > > > > I think they are not unit tests and should go to a different
> > > > > (performance?) test
> > > > > suite. Right now we don't have one but it seems to be time to create
it
> > > >
> > > > Of course, it's not unit tests, but my suggestion was based on our
> > > > testing convention.
> > > > What do you think about <modulename>/src/tests/performance ?
> > > >
> > > > Thanks,
> > > > Vladimir.
> > > >
> > > > > Thanks,
> > > > > Mikhail
> > > > >
> > > > > 2006/6/20, Vladimir Strigun <vstrigun@gmail.com>:
> > > > > > Hi Mikhail,
> > > > > >
> > > > > > AFAIK, we haven't such kind of tests in svn yet. It's hard to
define
> > > > > > best place for it, but since this suite is intended for java.math
> > > > > > testing only and it's implementation-independent tests, I believe
> > > > > > modules/math/src/test/java/tests/api is appropriate place. If
you
> > > > > > agree with this I will create patch for suite, add copyright
and
> > > > > > change package definition of classes.
> > > > > >
> > > > > > What do you think about it?
> > > > > >
> > > > > > Thanks,
> > > > > > Vladimir.
> > > > > >
> > > > > > On 6/13/06, Mikhail Loenko <mloenko@gmail.com> wrote:
> > > > > > > Hi Vladimir
> > > > > > >
> > > > > > > What do you think the most appropriate location and suite
for the tests
> > > > > > > in the HARMONY-551 are?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Mikhail
> > > > > > >
> > > > > > > 2006/6/2, Vladimir Strigun <vstrigun@gmail.com>:
> > > > > > > > Our team has done some analysis of current Harmony
implementation of
> > > > > > > > java.math package. The implementation was considered
from the
> > > > > > > > performance point of view and I'd like to share results
of our work
> > > > > > > > with you.
> > > > > > > >
> > > > > > > > The analysis and tuning was made from the enterprise-level
> > > > > > > > applications point of view which are known to use
BigInteger and
> > > > > > > > BigDecimal for storing numeric information. In most
cases the numbers
> > > > > > > > there fit well within 32 bits. So coming from the
BigDecimal
> > > > > > > > perspective which is really important for these applications
and
> > > > > > > > taking into account some specifics (small numbers)
we made some
> > > > > > > > optimizations in both BigDecimal and BigInteger. The
latter was tuned
> > > > > > > > specifically for BigDecimal:
> > > > > > > >
> > > > > > > > - Special handling for small numbers (fit within 32
bit) was added to
> > > > > > > > all methods
> > > > > > > > - Frequently used constants (0..10) were cached and
reused by valueOf
> > > > > > > > method (no need to create a new instance of BigInteger)
> > > > > > > > - as well as were powers of 10 (0..10)
> > > > > > > > - methods add(), divide(), divideAndRemainer() in
BigInteger were
> > > > > > > > optimized for short values if both arguments can fit
in 32 bits the
> > > > > > > > resulting BigInteger is created  by valueOf method.
> > > > > > > >
> > > > > > > >
> > > > > > > > If we consider enterprise level applications, we can
imagine that
> > > > > > > > toString() method is also frequently used. The method
was analyzed and
> > > > > > > > as a result we combined toString() methods in BigDecimal
and
> > > > > > > > BigInteger to one unified method in BigInteger. Method
> > > > > > > > BigInteger.toDecimalScaledString(int scale) now  is
used from  both
> > > > > > > > BigInteger and BigDecimal.  This way allows reducing
amount of created
> > > > > > > > objects and data copying. In addition, size calculation
was modified
> > > > > > > > for resulting array. In the new implementation the
size is calculated
> > > > > > > > with less precision. Because allocated char array
will be copied into
> > > > > > > > String and collected by GC after toString() then it
is not a problem
> > > > > > > > if the allocated char array will be slightly bigger
then necessary.
> > > > > > > >
> > > > > > > > I've attached the changes we made for BigInteger and
BigDecimal to Harmony-551
> > > > > > > >
> > > > > > > > We also have created a set of micro benchmarks (which
I'll to attach
> > > > > > > > to JIRA as well) which shows that our special-case
handling doesn't
> > > > > > > > break the performance for other cases and we do not
degrade in common,
> > > > > > > > and, of course, all unit tests pass with new version.
Below you can
> > > > > > > > find several comparisons in performance between current
version and
> > > > > > > > the fixed one.
> > > > > > > >
> > > > > > > > ===
> > > > > > > >
> > > > > > > > Ops/sec for toString() method of BigDecimal
> > > > > > > >
> > > > > > > > Number     Current       fixed one
> > > > > > > > of digits     version
> > > > > > > > 2       1121    5354
> > > > > > > > 4       774     7514
> > > > > > > > 8       615     6748
> > > > > > > > 12      743     5543
> > > > > > > > 16      623     4494
> > > > > > > > 24      389     4895
> > > > > > > > 32      306     3496
> > > > > > > > 48      232     5815
> > > > > > > > 64      224     3761
> > > > > > > > 128     91      87
> > > > > > > >
> > > > > > > > Ops/sec for divide method of BigInteger
> > > > > > > >
> > > > > > > > Number     Current       fixed one
> > > > > > > > of digits     version
> > > > > > > >
> > > > > > > > 2       5247    6315
> > > > > > > > 4       4623    6497
> > > > > > > > 8       5560    7491
> > > > > > > > 12      838     838
> > > > > > > > 16      2533    2063
> > > > > > > > 24      1689    1717
> > > > > > > > 32      2397    2494
> > > > > > > > 48      2143    2131
> > > > > > > > 64      613     525
> > > > > > > > 128     1399    1418
> > > > > > > >
> > > > > > > > Ops/sec for subtract method of BigInteger
> > > > > > > >
> > > > > > > > Number     Current       fixed one
> > > > > > > > of digits     version
> > > > > > > >
> > > > > > > > 2       3920    4394
> > > > > > > > 4       3300    3302
> > > > > > > > 8       5178    5640
> > > > > > > > 12      957     913
> > > > > > > > 16      3794    2904
> > > > > > > > 24      2057    1962
> > > > > > > > 32      3421    3241
> > > > > > > > 48      3469    2828
> > > > > > > > 64      652     610
> > > > > > > > 128     2318    2246
> > > > > > > >
> > > > > > > > ===
> > > > > > > >
> > > > > > > > Unfortunately we haven't look thoroughly to ITC contribution,
so it
> > > > > > > > may happen that some of the optimizations described
in this letter
> > > > > > > > were already implemented there. Daniel, could you
please consider our
> > > > > > > > new implementation when you start to merge implementations
math
> > > > > > > > packages. If optimization methods described above
already have been
> > > > > > > > implemented in ITC contribution it will be great to
compare internal
> > > > > > > > representation of BigInteger and try to measure affect
of different
> > > > > > > > approaches.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On 6/2/06, Vladimir Strigun (JIRA) <jira@apache.org>
wrote:
> > > > > > > > >     [ http://issues.apache.org/jira/browse/HARMONY-551?page=all
]
> > > > > > > > >
> > > > > > > > > Vladimir Strigun updated HARMONY-551:
> > > > > > > > > -------------------------------------
> > > > > > > > >
> > > > > > > > >    Attachment: Harmony-551.diffs
> > > > > > > > >
> > > > > > > > > Please try my patch.
> > > > > > > > > Several features were added:
> > > > > > > > > - special handling for small value
> > > > > > > > > - frequently used constans were cached
> > > > > > > > > - several methods were modified and optimized
for small value.
> > > > > > > > > etc.
> > > > > > > > >
> > > > > > > > > > [classlib][java.math] performance improvement
for java.math package
> > > > > > > > > > -------------------------------------------------------------------
> > > > > > > > > >
> > > > > > > > > >          Key: HARMONY-551
> > > > > > > > > >          URL: http://issues.apache.org/jira/browse/HARMONY-551
> > > > > > > > > >      Project: Harmony
> > > > > > > > > >         Type: Improvement
> > > > > > > > >
> > > > > > > > > >   Components: Classlib
> > > > > > > > > >     Reporter: Vladimir Strigun
> > > > > > > > > >  Attachments: Harmony-551.diffs
> > > > > > > > > >
> > > > > > > > > > Performance improvement for BigDecimal and
BigInteger classes.
> > > > > > > > > > I will attach patch soon.
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > This message is automatically generated by JIRA.
> > > > > > > > > -
> > > > > > > > > If you think it was sent incorrectly contact
one of the administrators:
> > > > > > > > >   http://issues.apache.org/jira/secure/Administrators.jspa
> > > > > > > > > -
> > > > > > > > > For more information on JIRA, see:
> > > > > > > > >   http://www.atlassian.com/software/jira
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > ---------------------------------------------------------------------
> > > > > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > > > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > ---------------------------------------------------------------------
> > > > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > ---------------------------------------------------------------------
> > > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > > > >
> > > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > > >
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > >
> > > >
> > >
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message