harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mikhail Loenko" <mloe...@gmail.com>
Subject Re: [classlib][math] Location of performance tests (was: Re: performance improvement for java.math package)
Date Mon, 26 Jun 2006 02:17:35 GMT
Agreed. What I suggested is calculate "golden" values on the fly, so you'll
have golden values for the given machine, VM, etc

Anyway if you feel the tests should not go to any suite, I'll just
close the JIRA issue.

Thanks,
Mikhail

2006/6/23, Vladimir Strigun <vstrigun@gmail.com>:
> Unfortunately it is not a regression test. It's just micro-benchmark
> for measure performance of particular methods from BigInteger and
> BigDecimal classes.
> So, the result of execution is ops/sec value for every method, and
> since it dependent on machine speed, VM, etc, comparison between
> actual and 'golden' values might be incorrect.
>
> Thanks,
> Vladimir.
>
> On 6/22/06, Mikhail Loenko <mloenko@gmail.com> wrote:
> > The first thing that came into my mind is as far as it is a regression test,
> > put somewhere next to the test both old and new Math implementations,
> > measure 'golden' performances and than measure where the current performance
> > in comparison to the golden is.
> >
> > This scenario might be simplified. If your optimization works e.g. for numbers
> > less than say 1'000'000, you can compare performance for 999'999 and
> > 1'000'001
> >
> > Thanks,
> > Mikhail
> >
> > 2006/6/22, Vladimir Strigun <vstrigun@gmail.com>:
> > > Mikhail,
> > >
> > > I can convert it to JUnit, but I'm not pretty sure about returning
> > > pass/fail. When you think test should return fail?
> > > Results of test execution can be different on different VM's, it also
> > > dependent of machine speed, etc.
> > >
> > > Thanks,
> > > Vladimir.
> > >
> > > On 6/22/06, Mikhail Loenko <mloenko@gmail.com> wrote:
> > > > I've confused all the things. Sorry. Of course the tests should go to
> > > > math/src/test/performance
> > > >
> > > > Vladimir, is it possible to convert the test to JUnit format and make
them
> > > > report pass/fail status rather than printing log?
> > > >
> > > > Thanks,
> > > > Mikhail
> > > >
> > > > 2006/6/21, Mikhail Loenko <mloenko@gmail.com>:
> > > > > I'd like to bring it to common judgment.
> > > > >
> > > > > AFAIU we have two basic options for performance tests location: make
> > > > > them module level or make a top-level tests subtree that would contain
> > > > > all types of the tests except for unit tests
> > > > >
> > > > > BTW, In the testing convention my intent was to cover unit tests
only.
> > > > > Though we did not agree which exactly tests are "unit", so I avoided
that word.
> > > > > But I definitely did not mean performance, stress and whatever special
types
> > > > > of the tests. If no one objects I'll add some clarification to the
conventions
> > > > > proposal.
> > > > >
> > > > > Thanks,
> > > > > Mikhail
> > > > >
> > > > > 2006/6/20, Vladimir Strigun <vstrigun@gmail.com>:
> > > > > > On 6/20/06, Mikhail Loenko <mloenko@gmail.com> wrote:
> > > > > > > I think they are not unit tests and should go to a different
> > > > > > > (performance?) test
> > > > > > > suite. Right now we don't have one but it seems to be time
to create it
> > > > > >
> > > > > > Of course, it's not unit tests, but my suggestion was based
on our
> > > > > > testing convention.
> > > > > > What do you think about <modulename>/src/tests/performance
?
> > > > > >
> > > > > > Thanks,
> > > > > > Vladimir.
> > > > > >
> > > > > > > Thanks,
> > > > > > > Mikhail
> > > > > > >
> > > > > > > 2006/6/20, Vladimir Strigun <vstrigun@gmail.com>:
> > > > > > > > Hi Mikhail,
> > > > > > > >
> > > > > > > > AFAIK, we haven't such kind of tests in svn yet. It's
hard to define
> > > > > > > > best place for it, but since this suite is intended
for java.math
> > > > > > > > testing only and it's implementation-independent tests,
I believe
> > > > > > > > modules/math/src/test/java/tests/api is appropriate
place. If you
> > > > > > > > agree with this I will create patch for suite, add
copyright and
> > > > > > > > change package definition of classes.
> > > > > > > >
> > > > > > > > What do you think about it?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Vladimir.
> > > > > > > >
> > > > > > > > On 6/13/06, Mikhail Loenko <mloenko@gmail.com>
wrote:
> > > > > > > > > Hi Vladimir
> > > > > > > > >
> > > > > > > > > What do you think the most appropriate location
and suite for the tests
> > > > > > > > > in the HARMONY-551 are?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Mikhail
> > > > > > > > >
> > > > > > > > > 2006/6/2, Vladimir Strigun <vstrigun@gmail.com>:
> > > > > > > > > > Our team has done some analysis of current
Harmony implementation of
> > > > > > > > > > java.math package. The implementation was
considered from the
> > > > > > > > > > performance point of view and I'd like to
share results of our work
> > > > > > > > > > with you.
> > > > > > > > > >
> > > > > > > > > > The analysis and tuning was made from the
enterprise-level
> > > > > > > > > > applications point of view which are known
to use BigInteger and
> > > > > > > > > > BigDecimal for storing numeric information.
In most cases the numbers
> > > > > > > > > > there fit well within 32 bits. So coming
from the BigDecimal
> > > > > > > > > > perspective which is really important for
these applications and
> > > > > > > > > > taking into account some specifics (small
numbers) we made some
> > > > > > > > > > optimizations in both BigDecimal and BigInteger.
The latter was tuned
> > > > > > > > > > specifically for BigDecimal:
> > > > > > > > > >
> > > > > > > > > > - Special handling for small numbers (fit
within 32 bit) was added to
> > > > > > > > > > all methods
> > > > > > > > > > - Frequently used constants (0..10) were
cached and reused by valueOf
> > > > > > > > > > method (no need to create a new instance
of BigInteger)
> > > > > > > > > > - as well as were powers of 10 (0..10)
> > > > > > > > > > - methods add(), divide(), divideAndRemainer()
in BigInteger were
> > > > > > > > > > optimized for short values if both arguments
can fit in 32 bits the
> > > > > > > > > > resulting BigInteger is created  by valueOf
method.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > If we consider enterprise level applications,
we can imagine that
> > > > > > > > > > toString() method is also frequently used.
The method was analyzed and
> > > > > > > > > > as a result we combined toString() methods
in BigDecimal and
> > > > > > > > > > BigInteger to one unified method in BigInteger.
Method
> > > > > > > > > > BigInteger.toDecimalScaledString(int scale)
now  is used from  both
> > > > > > > > > > BigInteger and BigDecimal.  This way allows
reducing amount of created
> > > > > > > > > > objects and data copying. In addition, size
calculation was modified
> > > > > > > > > > for resulting array. In the new implementation
the size is calculated
> > > > > > > > > > with less precision. Because allocated char
array will be copied into
> > > > > > > > > > String and collected by GC after toString()
then it is not a problem
> > > > > > > > > > if the allocated char array will be slightly
bigger then necessary.
> > > > > > > > > >
> > > > > > > > > > I've attached the changes we made for BigInteger
and BigDecimal to Harmony-551
> > > > > > > > > >
> > > > > > > > > > We also have created a set of micro benchmarks
(which I'll to attach
> > > > > > > > > > to JIRA as well) which shows that our special-case
handling doesn't
> > > > > > > > > > break the performance for other cases and
we do not degrade in common,
> > > > > > > > > > and, of course, all unit tests pass with
new version. Below you can
> > > > > > > > > > find several comparisons in performance
between current version and
> > > > > > > > > > the fixed one.
> > > > > > > > > >
> > > > > > > > > > ===
> > > > > > > > > >
> > > > > > > > > > Ops/sec for toString() method of BigDecimal
> > > > > > > > > >
> > > > > > > > > > Number     Current       fixed one
> > > > > > > > > > of digits     version
> > > > > > > > > > 2       1121    5354
> > > > > > > > > > 4       774     7514
> > > > > > > > > > 8       615     6748
> > > > > > > > > > 12      743     5543
> > > > > > > > > > 16      623     4494
> > > > > > > > > > 24      389     4895
> > > > > > > > > > 32      306     3496
> > > > > > > > > > 48      232     5815
> > > > > > > > > > 64      224     3761
> > > > > > > > > > 128     91      87
> > > > > > > > > >
> > > > > > > > > > Ops/sec for divide method of BigInteger
> > > > > > > > > >
> > > > > > > > > > Number     Current       fixed one
> > > > > > > > > > of digits     version
> > > > > > > > > >
> > > > > > > > > > 2       5247    6315
> > > > > > > > > > 4       4623    6497
> > > > > > > > > > 8       5560    7491
> > > > > > > > > > 12      838     838
> > > > > > > > > > 16      2533    2063
> > > > > > > > > > 24      1689    1717
> > > > > > > > > > 32      2397    2494
> > > > > > > > > > 48      2143    2131
> > > > > > > > > > 64      613     525
> > > > > > > > > > 128     1399    1418
> > > > > > > > > >
> > > > > > > > > > Ops/sec for subtract method of BigInteger
> > > > > > > > > >
> > > > > > > > > > Number     Current       fixed one
> > > > > > > > > > of digits     version
> > > > > > > > > >
> > > > > > > > > > 2       3920    4394
> > > > > > > > > > 4       3300    3302
> > > > > > > > > > 8       5178    5640
> > > > > > > > > > 12      957     913
> > > > > > > > > > 16      3794    2904
> > > > > > > > > > 24      2057    1962
> > > > > > > > > > 32      3421    3241
> > > > > > > > > > 48      3469    2828
> > > > > > > > > > 64      652     610
> > > > > > > > > > 128     2318    2246
> > > > > > > > > >
> > > > > > > > > > ===
> > > > > > > > > >
> > > > > > > > > > Unfortunately we haven't look thoroughly
to ITC contribution, so it
> > > > > > > > > > may happen that some of the optimizations
described in this letter
> > > > > > > > > > were already implemented there. Daniel,
could you please consider our
> > > > > > > > > > new implementation when you start to merge
implementations math
> > > > > > > > > > packages. If optimization methods described
above already have been
> > > > > > > > > > implemented in ITC contribution it will
be great to compare internal
> > > > > > > > > > representation of BigInteger and try to
measure affect of different
> > > > > > > > > > approaches.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On 6/2/06, Vladimir Strigun (JIRA) <jira@apache.org>
wrote:
> > > > > > > > > > >     [ http://issues.apache.org/jira/browse/HARMONY-551?page=all
]
> > > > > > > > > > >
> > > > > > > > > > > Vladimir Strigun updated HARMONY-551:
> > > > > > > > > > > -------------------------------------
> > > > > > > > > > >
> > > > > > > > > > >    Attachment: Harmony-551.diffs
> > > > > > > > > > >
> > > > > > > > > > > Please try my patch.
> > > > > > > > > > > Several features were added:
> > > > > > > > > > > - special handling for small value
> > > > > > > > > > > - frequently used constans were cached
> > > > > > > > > > > - several methods were modified and
optimized for small value.
> > > > > > > > > > > etc.
> > > > > > > > > > >
> > > > > > > > > > > > [classlib][java.math] performance
improvement for java.math package
> > > > > > > > > > > > -------------------------------------------------------------------
> > > > > > > > > > > >
> > > > > > > > > > > >          Key: HARMONY-551
> > > > > > > > > > > >          URL: http://issues.apache.org/jira/browse/HARMONY-551
> > > > > > > > > > > >      Project: Harmony
> > > > > > > > > > > >         Type: Improvement
> > > > > > > > > > >
> > > > > > > > > > > >   Components: Classlib
> > > > > > > > > > > >     Reporter: Vladimir Strigun
> > > > > > > > > > > >  Attachments: Harmony-551.diffs
> > > > > > > > > > > >
> > > > > > > > > > > > Performance improvement for BigDecimal
and BigInteger classes.
> > > > > > > > > > > > I will attach patch soon.
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > This message is automatically generated
by JIRA.
> > > > > > > > > > > -
> > > > > > > > > > > If you think it was sent incorrectly
contact one of the administrators:
> > > > > > > > > > >   http://issues.apache.org/jira/secure/Administrators.jspa
> > > > > > > > > > > -
> > > > > > > > > > > For more information on JIRA, see:
> > > > > > > > > > >   http://www.atlassian.com/software/jira
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > ---------------------------------------------------------------------
> > > > > > > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > > > > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > > > > > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > ---------------------------------------------------------------------
> > > > > > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > > > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > > > > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > ---------------------------------------------------------------------
> > > > > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > > > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > ---------------------------------------------------------------------
> > > > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > ---------------------------------------------------------------------
> > > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > > > >
> > > > > >
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message