harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Strigun" <vstri...@gmail.com>
Subject Re: [classlib][math] Location of performance tests (was: Re: performance improvement for java.math package)
Date Fri, 23 Jun 2006 11:36:18 GMT
Unfortunately it is not a regression test. It's just micro-benchmark
for measure performance of particular methods from BigInteger and
BigDecimal classes.
So, the result of execution is ops/sec value for every method, and
since it dependent on machine speed, VM, etc, comparison between
actual and 'golden' values might be incorrect.

Thanks,
Vladimir.

On 6/22/06, Mikhail Loenko <mloenko@gmail.com> wrote:
> The first thing that came into my mind is as far as it is a regression test,
> put somewhere next to the test both old and new Math implementations,
> measure 'golden' performances and than measure where the current performance
> in comparison to the golden is.
>
> This scenario might be simplified. If your optimization works e.g. for numbers
> less than say 1'000'000, you can compare performance for 999'999 and
> 1'000'001
>
> Thanks,
> Mikhail
>
> 2006/6/22, Vladimir Strigun <vstrigun@gmail.com>:
> > Mikhail,
> >
> > I can convert it to JUnit, but I'm not pretty sure about returning
> > pass/fail. When you think test should return fail?
> > Results of test execution can be different on different VM's, it also
> > dependent of machine speed, etc.
> >
> > Thanks,
> > Vladimir.
> >
> > On 6/22/06, Mikhail Loenko <mloenko@gmail.com> wrote:
> > > I've confused all the things. Sorry. Of course the tests should go to
> > > math/src/test/performance
> > >
> > > Vladimir, is it possible to convert the test to JUnit format and make them
> > > report pass/fail status rather than printing log?
> > >
> > > Thanks,
> > > Mikhail
> > >
> > > 2006/6/21, Mikhail Loenko <mloenko@gmail.com>:
> > > > I'd like to bring it to common judgment.
> > > >
> > > > AFAIU we have two basic options for performance tests location: make
> > > > them module level or make a top-level tests subtree that would contain
> > > > all types of the tests except for unit tests
> > > >
> > > > BTW, In the testing convention my intent was to cover unit tests only.
> > > > Though we did not agree which exactly tests are "unit", so I avoided that
word.
> > > > But I definitely did not mean performance, stress and whatever special
types
> > > > of the tests. If no one objects I'll add some clarification to the conventions
> > > > proposal.
> > > >
> > > > Thanks,
> > > > Mikhail
> > > >
> > > > 2006/6/20, Vladimir Strigun <vstrigun@gmail.com>:
> > > > > On 6/20/06, Mikhail Loenko <mloenko@gmail.com> wrote:
> > > > > > I think they are not unit tests and should go to a different
> > > > > > (performance?) test
> > > > > > suite. Right now we don't have one but it seems to be time to
create it
> > > > >
> > > > > Of course, it's not unit tests, but my suggestion was based on our
> > > > > testing convention.
> > > > > What do you think about <modulename>/src/tests/performance
?
> > > > >
> > > > > Thanks,
> > > > > Vladimir.
> > > > >
> > > > > > Thanks,
> > > > > > Mikhail
> > > > > >
> > > > > > 2006/6/20, Vladimir Strigun <vstrigun@gmail.com>:
> > > > > > > Hi Mikhail,
> > > > > > >
> > > > > > > AFAIK, we haven't such kind of tests in svn yet. It's hard
to define
> > > > > > > best place for it, but since this suite is intended for
java.math
> > > > > > > testing only and it's implementation-independent tests,
I believe
> > > > > > > modules/math/src/test/java/tests/api is appropriate place.
If you
> > > > > > > agree with this I will create patch for suite, add copyright
and
> > > > > > > change package definition of classes.
> > > > > > >
> > > > > > > What do you think about it?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Vladimir.
> > > > > > >
> > > > > > > On 6/13/06, Mikhail Loenko <mloenko@gmail.com> wrote:
> > > > > > > > Hi Vladimir
> > > > > > > >
> > > > > > > > What do you think the most appropriate location and
suite for the tests
> > > > > > > > in the HARMONY-551 are?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Mikhail
> > > > > > > >
> > > > > > > > 2006/6/2, Vladimir Strigun <vstrigun@gmail.com>:
> > > > > > > > > Our team has done some analysis of current Harmony
implementation of
> > > > > > > > > java.math package. The implementation was considered
from the
> > > > > > > > > performance point of view and I'd like to share
results of our work
> > > > > > > > > with you.
> > > > > > > > >
> > > > > > > > > The analysis and tuning was made from the enterprise-level
> > > > > > > > > applications point of view which are known to
use BigInteger and
> > > > > > > > > BigDecimal for storing numeric information. In
most cases the numbers
> > > > > > > > > there fit well within 32 bits. So coming from
the BigDecimal
> > > > > > > > > perspective which is really important for these
applications and
> > > > > > > > > taking into account some specifics (small numbers)
we made some
> > > > > > > > > optimizations in both BigDecimal and BigInteger.
The latter was tuned
> > > > > > > > > specifically for BigDecimal:
> > > > > > > > >
> > > > > > > > > - Special handling for small numbers (fit within
32 bit) was added to
> > > > > > > > > all methods
> > > > > > > > > - Frequently used constants (0..10) were cached
and reused by valueOf
> > > > > > > > > method (no need to create a new instance of BigInteger)
> > > > > > > > > - as well as were powers of 10 (0..10)
> > > > > > > > > - methods add(), divide(), divideAndRemainer()
in BigInteger were
> > > > > > > > > optimized for short values if both arguments
can fit in 32 bits the
> > > > > > > > > resulting BigInteger is created  by valueOf method.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > If we consider enterprise level applications,
we can imagine that
> > > > > > > > > toString() method is also frequently used. The
method was analyzed and
> > > > > > > > > as a result we combined toString() methods in
BigDecimal and
> > > > > > > > > BigInteger to one unified method in BigInteger.
Method
> > > > > > > > > BigInteger.toDecimalScaledString(int scale) now
 is used from  both
> > > > > > > > > BigInteger and BigDecimal.  This way allows reducing
amount of created
> > > > > > > > > objects and data copying. In addition, size calculation
was modified
> > > > > > > > > for resulting array. In the new implementation
the size is calculated
> > > > > > > > > with less precision. Because allocated char array
will be copied into
> > > > > > > > > String and collected by GC after toString() then
it is not a problem
> > > > > > > > > if the allocated char array will be slightly
bigger then necessary.
> > > > > > > > >
> > > > > > > > > I've attached the changes we made for BigInteger
and BigDecimal to Harmony-551
> > > > > > > > >
> > > > > > > > > We also have created a set of micro benchmarks
(which I'll to attach
> > > > > > > > > to JIRA as well) which shows that our special-case
handling doesn't
> > > > > > > > > break the performance for other cases and we
do not degrade in common,
> > > > > > > > > and, of course, all unit tests pass with new
version. Below you can
> > > > > > > > > find several comparisons in performance between
current version and
> > > > > > > > > the fixed one.
> > > > > > > > >
> > > > > > > > > ===
> > > > > > > > >
> > > > > > > > > Ops/sec for toString() method of BigDecimal
> > > > > > > > >
> > > > > > > > > Number     Current       fixed one
> > > > > > > > > of digits     version
> > > > > > > > > 2       1121    5354
> > > > > > > > > 4       774     7514
> > > > > > > > > 8       615     6748
> > > > > > > > > 12      743     5543
> > > > > > > > > 16      623     4494
> > > > > > > > > 24      389     4895
> > > > > > > > > 32      306     3496
> > > > > > > > > 48      232     5815
> > > > > > > > > 64      224     3761
> > > > > > > > > 128     91      87
> > > > > > > > >
> > > > > > > > > Ops/sec for divide method of BigInteger
> > > > > > > > >
> > > > > > > > > Number     Current       fixed one
> > > > > > > > > of digits     version
> > > > > > > > >
> > > > > > > > > 2       5247    6315
> > > > > > > > > 4       4623    6497
> > > > > > > > > 8       5560    7491
> > > > > > > > > 12      838     838
> > > > > > > > > 16      2533    2063
> > > > > > > > > 24      1689    1717
> > > > > > > > > 32      2397    2494
> > > > > > > > > 48      2143    2131
> > > > > > > > > 64      613     525
> > > > > > > > > 128     1399    1418
> > > > > > > > >
> > > > > > > > > Ops/sec for subtract method of BigInteger
> > > > > > > > >
> > > > > > > > > Number     Current       fixed one
> > > > > > > > > of digits     version
> > > > > > > > >
> > > > > > > > > 2       3920    4394
> > > > > > > > > 4       3300    3302
> > > > > > > > > 8       5178    5640
> > > > > > > > > 12      957     913
> > > > > > > > > 16      3794    2904
> > > > > > > > > 24      2057    1962
> > > > > > > > > 32      3421    3241
> > > > > > > > > 48      3469    2828
> > > > > > > > > 64      652     610
> > > > > > > > > 128     2318    2246
> > > > > > > > >
> > > > > > > > > ===
> > > > > > > > >
> > > > > > > > > Unfortunately we haven't look thoroughly to ITC
contribution, so it
> > > > > > > > > may happen that some of the optimizations described
in this letter
> > > > > > > > > were already implemented there. Daniel, could
you please consider our
> > > > > > > > > new implementation when you start to merge implementations
math
> > > > > > > > > packages. If optimization methods described above
already have been
> > > > > > > > > implemented in ITC contribution it will be great
to compare internal
> > > > > > > > > representation of BigInteger and try to measure
affect of different
> > > > > > > > > approaches.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On 6/2/06, Vladimir Strigun (JIRA) <jira@apache.org>
wrote:
> > > > > > > > > >     [ http://issues.apache.org/jira/browse/HARMONY-551?page=all
]
> > > > > > > > > >
> > > > > > > > > > Vladimir Strigun updated HARMONY-551:
> > > > > > > > > > -------------------------------------
> > > > > > > > > >
> > > > > > > > > >    Attachment: Harmony-551.diffs
> > > > > > > > > >
> > > > > > > > > > Please try my patch.
> > > > > > > > > > Several features were added:
> > > > > > > > > > - special handling for small value
> > > > > > > > > > - frequently used constans were cached
> > > > > > > > > > - several methods were modified and optimized
for small value.
> > > > > > > > > > etc.
> > > > > > > > > >
> > > > > > > > > > > [classlib][java.math] performance improvement
for java.math package
> > > > > > > > > > > -------------------------------------------------------------------
> > > > > > > > > > >
> > > > > > > > > > >          Key: HARMONY-551
> > > > > > > > > > >          URL: http://issues.apache.org/jira/browse/HARMONY-551
> > > > > > > > > > >      Project: Harmony
> > > > > > > > > > >         Type: Improvement
> > > > > > > > > >
> > > > > > > > > > >   Components: Classlib
> > > > > > > > > > >     Reporter: Vladimir Strigun
> > > > > > > > > > >  Attachments: Harmony-551.diffs
> > > > > > > > > > >
> > > > > > > > > > > Performance improvement for BigDecimal
and BigInteger classes.
> > > > > > > > > > > I will attach patch soon.
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > This message is automatically generated
by JIRA.
> > > > > > > > > > -
> > > > > > > > > > If you think it was sent incorrectly contact
one of the administrators:
> > > > > > > > > >   http://issues.apache.org/jira/secure/Administrators.jspa
> > > > > > > > > > -
> > > > > > > > > > For more information on JIRA, see:
> > > > > > > > > >   http://www.atlassian.com/software/jira
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > ---------------------------------------------------------------------
> > > > > > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > > > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > > > > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > ---------------------------------------------------------------------
> > > > > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > > > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > ---------------------------------------------------------------------
> > > > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > ---------------------------------------------------------------------
> > > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > > > >
> > > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > > >
> > > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message