harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Strigun" <vstri...@gmail.com>
Subject Re: [classlib][math] Location of performance tests (was: Re: performance improvement for java.math package)
Date Wed, 28 Jun 2006 12:38:36 GMT
I think it's better to close Harmony-551. When Daniel and his team
will prepare new version of java.math we can compare implementation
using this suite and then decide, whether we need to put it into svn.

Thanks,
Vladimir.

On 6/26/06, Mikhail Loenko <mloenko@gmail.com> wrote:
> Agreed. What I suggested is calculate "golden" values on the fly, so you'll
> have golden values for the given machine, VM, etc
>
> Anyway if you feel the tests should not go to any suite, I'll just
> close the JIRA issue.
>
> Thanks,
> Mikhail
>
> 2006/6/23, Vladimir Strigun <vstrigun@gmail.com>:
> > Unfortunately it is not a regression test. It's just micro-benchmark
> > for measure performance of particular methods from BigInteger and
> > BigDecimal classes.
> > So, the result of execution is ops/sec value for every method, and
> > since it dependent on machine speed, VM, etc, comparison between
> > actual and 'golden' values might be incorrect.
> >
> > Thanks,
> > Vladimir.
> >
> > On 6/22/06, Mikhail Loenko <mloenko@gmail.com> wrote:
> > > The first thing that came into my mind is as far as it is a regression test,
> > > put somewhere next to the test both old and new Math implementations,
> > > measure 'golden' performances and than measure where the current performance
> > > in comparison to the golden is.
> > >
> > > This scenario might be simplified. If your optimization works e.g. for numbers
> > > less than say 1'000'000, you can compare performance for 999'999 and
> > > 1'000'001
> > >
> > > Thanks,
> > > Mikhail
> > >
> > > 2006/6/22, Vladimir Strigun <vstrigun@gmail.com>:
> > > > Mikhail,
> > > >
> > > > I can convert it to JUnit, but I'm not pretty sure about returning
> > > > pass/fail. When you think test should return fail?
> > > > Results of test execution can be different on different VM's, it also
> > > > dependent of machine speed, etc.
> > > >
> > > > Thanks,
> > > > Vladimir.
> > > >
> > > > On 6/22/06, Mikhail Loenko <mloenko@gmail.com> wrote:
> > > > > I've confused all the things. Sorry. Of course the tests should go
to
> > > > > math/src/test/performance
> > > > >
> > > > > Vladimir, is it possible to convert the test to JUnit format and
make them
> > > > > report pass/fail status rather than printing log?
> > > > >
> > > > > Thanks,
> > > > > Mikhail
> > > > >
> > > > > 2006/6/21, Mikhail Loenko <mloenko@gmail.com>:
> > > > > > I'd like to bring it to common judgment.
> > > > > >
> > > > > > AFAIU we have two basic options for performance tests location:
make
> > > > > > them module level or make a top-level tests subtree that would
contain
> > > > > > all types of the tests except for unit tests
> > > > > >
> > > > > > BTW, In the testing convention my intent was to cover unit tests
only.
> > > > > > Though we did not agree which exactly tests are "unit", so I
avoided that word.
> > > > > > But I definitely did not mean performance, stress and whatever
special types
> > > > > > of the tests. If no one objects I'll add some clarification
to the conventions
> > > > > > proposal.
> > > > > >
> > > > > > Thanks,
> > > > > > Mikhail
> > > > > >
> > > > > > 2006/6/20, Vladimir Strigun <vstrigun@gmail.com>:
> > > > > > > On 6/20/06, Mikhail Loenko <mloenko@gmail.com> wrote:
> > > > > > > > I think they are not unit tests and should go to a
different
> > > > > > > > (performance?) test
> > > > > > > > suite. Right now we don't have one but it seems to
be time to create it
> > > > > > >
> > > > > > > Of course, it's not unit tests, but my suggestion was based
on our
> > > > > > > testing convention.
> > > > > > > What do you think about <modulename>/src/tests/performance
?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Vladimir.
> > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Mikhail
> > > > > > > >
> > > > > > > > 2006/6/20, Vladimir Strigun <vstrigun@gmail.com>:
> > > > > > > > > Hi Mikhail,
> > > > > > > > >
> > > > > > > > > AFAIK, we haven't such kind of tests in svn yet.
It's hard to define
> > > > > > > > > best place for it, but since this suite is intended
for java.math
> > > > > > > > > testing only and it's implementation-independent
tests, I believe
> > > > > > > > > modules/math/src/test/java/tests/api is appropriate
place. If you
> > > > > > > > > agree with this I will create patch for suite,
add copyright and
> > > > > > > > > change package definition of classes.
> > > > > > > > >
> > > > > > > > > What do you think about it?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Vladimir.
> > > > > > > > >
> > > > > > > > > On 6/13/06, Mikhail Loenko <mloenko@gmail.com>
wrote:
> > > > > > > > > > Hi Vladimir
> > > > > > > > > >
> > > > > > > > > > What do you think the most appropriate location
and suite for the tests
> > > > > > > > > > in the HARMONY-551 are?
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > Mikhail
> > > > > > > > > >
> > > > > > > > > > 2006/6/2, Vladimir Strigun <vstrigun@gmail.com>:
> > > > > > > > > > > Our team has done some analysis of
current Harmony implementation of
> > > > > > > > > > > java.math package. The implementation
was considered from the
> > > > > > > > > > > performance point of view and I'd like
to share results of our work
> > > > > > > > > > > with you.
> > > > > > > > > > >
> > > > > > > > > > > The analysis and tuning was made from
the enterprise-level
> > > > > > > > > > > applications point of view which are
known to use BigInteger and
> > > > > > > > > > > BigDecimal for storing numeric information.
In most cases the numbers
> > > > > > > > > > > there fit well within 32 bits. So coming
from the BigDecimal
> > > > > > > > > > > perspective which is really important
for these applications and
> > > > > > > > > > > taking into account some specifics
(small numbers) we made some
> > > > > > > > > > > optimizations in both BigDecimal and
BigInteger. The latter was tuned
> > > > > > > > > > > specifically for BigDecimal:
> > > > > > > > > > >
> > > > > > > > > > > - Special handling for small numbers
(fit within 32 bit) was added to
> > > > > > > > > > > all methods
> > > > > > > > > > > - Frequently used constants (0..10)
were cached and reused by valueOf
> > > > > > > > > > > method (no need to create a new instance
of BigInteger)
> > > > > > > > > > > - as well as were powers of 10 (0..10)
> > > > > > > > > > > - methods add(), divide(), divideAndRemainer()
in BigInteger were
> > > > > > > > > > > optimized for short values if both
arguments can fit in 32 bits the
> > > > > > > > > > > resulting BigInteger is created  by
valueOf method.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > If we consider enterprise level applications,
we can imagine that
> > > > > > > > > > > toString() method is also frequently
used. The method was analyzed and
> > > > > > > > > > > as a result we combined toString()
methods in BigDecimal and
> > > > > > > > > > > BigInteger to one unified method in
BigInteger. Method
> > > > > > > > > > > BigInteger.toDecimalScaledString(int
scale) now  is used from  both
> > > > > > > > > > > BigInteger and BigDecimal.  This way
allows reducing amount of created
> > > > > > > > > > > objects and data copying. In addition,
size calculation was modified
> > > > > > > > > > > for resulting array. In the new implementation
the size is calculated
> > > > > > > > > > > with less precision. Because allocated
char array will be copied into
> > > > > > > > > > > String and collected by GC after toString()
then it is not a problem
> > > > > > > > > > > if the allocated char array will be
slightly bigger then necessary.
> > > > > > > > > > >
> > > > > > > > > > > I've attached the changes we made for
BigInteger and BigDecimal to Harmony-551
> > > > > > > > > > >
> > > > > > > > > > > We also have created a set of micro
benchmarks (which I'll to attach
> > > > > > > > > > > to JIRA as well) which shows that our
special-case handling doesn't
> > > > > > > > > > > break the performance for other cases
and we do not degrade in common,
> > > > > > > > > > > and, of course, all unit tests pass
with new version. Below you can
> > > > > > > > > > > find several comparisons in performance
between current version and
> > > > > > > > > > > the fixed one.
> > > > > > > > > > >
> > > > > > > > > > > ===
> > > > > > > > > > >
> > > > > > > > > > > Ops/sec for toString() method of BigDecimal
> > > > > > > > > > >
> > > > > > > > > > > Number     Current       fixed one
> > > > > > > > > > > of digits     version
> > > > > > > > > > > 2       1121    5354
> > > > > > > > > > > 4       774     7514
> > > > > > > > > > > 8       615     6748
> > > > > > > > > > > 12      743     5543
> > > > > > > > > > > 16      623     4494
> > > > > > > > > > > 24      389     4895
> > > > > > > > > > > 32      306     3496
> > > > > > > > > > > 48      232     5815
> > > > > > > > > > > 64      224     3761
> > > > > > > > > > > 128     91      87
> > > > > > > > > > >
> > > > > > > > > > > Ops/sec for divide method of BigInteger
> > > > > > > > > > >
> > > > > > > > > > > Number     Current       fixed one
> > > > > > > > > > > of digits     version
> > > > > > > > > > >
> > > > > > > > > > > 2       5247    6315
> > > > > > > > > > > 4       4623    6497
> > > > > > > > > > > 8       5560    7491
> > > > > > > > > > > 12      838     838
> > > > > > > > > > > 16      2533    2063
> > > > > > > > > > > 24      1689    1717
> > > > > > > > > > > 32      2397    2494
> > > > > > > > > > > 48      2143    2131
> > > > > > > > > > > 64      613     525
> > > > > > > > > > > 128     1399    1418
> > > > > > > > > > >
> > > > > > > > > > > Ops/sec for subtract method of BigInteger
> > > > > > > > > > >
> > > > > > > > > > > Number     Current       fixed one
> > > > > > > > > > > of digits     version
> > > > > > > > > > >
> > > > > > > > > > > 2       3920    4394
> > > > > > > > > > > 4       3300    3302
> > > > > > > > > > > 8       5178    5640
> > > > > > > > > > > 12      957     913
> > > > > > > > > > > 16      3794    2904
> > > > > > > > > > > 24      2057    1962
> > > > > > > > > > > 32      3421    3241
> > > > > > > > > > > 48      3469    2828
> > > > > > > > > > > 64      652     610
> > > > > > > > > > > 128     2318    2246
> > > > > > > > > > >
> > > > > > > > > > > ===
> > > > > > > > > > >
> > > > > > > > > > > Unfortunately we haven't look thoroughly
to ITC contribution, so it
> > > > > > > > > > > may happen that some of the optimizations
described in this letter
> > > > > > > > > > > were already implemented there. Daniel,
could you please consider our
> > > > > > > > > > > new implementation when you start to
merge implementations math
> > > > > > > > > > > packages. If optimization methods described
above already have been
> > > > > > > > > > > implemented in ITC contribution it
will be great to compare internal
> > > > > > > > > > > representation of BigInteger and try
to measure affect of different
> > > > > > > > > > > approaches.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On 6/2/06, Vladimir Strigun (JIRA)
<jira@apache.org> wrote:
> > > > > > > > > > > >     [ http://issues.apache.org/jira/browse/HARMONY-551?page=all
]
> > > > > > > > > > > >
> > > > > > > > > > > > Vladimir Strigun updated HARMONY-551:
> > > > > > > > > > > > -------------------------------------
> > > > > > > > > > > >
> > > > > > > > > > > >    Attachment: Harmony-551.diffs
> > > > > > > > > > > >
> > > > > > > > > > > > Please try my patch.
> > > > > > > > > > > > Several features were added:
> > > > > > > > > > > > - special handling for small value
> > > > > > > > > > > > - frequently used constans were
cached
> > > > > > > > > > > > - several methods were modified
and optimized for small value.
> > > > > > > > > > > > etc.
> > > > > > > > > > > >
> > > > > > > > > > > > > [classlib][java.math] performance
improvement for java.math package
> > > > > > > > > > > > > -------------------------------------------------------------------
> > > > > > > > > > > > >
> > > > > > > > > > > > >          Key: HARMONY-551
> > > > > > > > > > > > >          URL: http://issues.apache.org/jira/browse/HARMONY-551
> > > > > > > > > > > > >      Project: Harmony
> > > > > > > > > > > > >         Type: Improvement
> > > > > > > > > > > >
> > > > > > > > > > > > >   Components: Classlib
> > > > > > > > > > > > >     Reporter: Vladimir Strigun
> > > > > > > > > > > > >  Attachments: Harmony-551.diffs
> > > > > > > > > > > > >
> > > > > > > > > > > > > Performance improvement for
BigDecimal and BigInteger classes.
> > > > > > > > > > > > > I will attach patch soon.
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > > This message is automatically
generated by JIRA.
> > > > > > > > > > > > -
> > > > > > > > > > > > If you think it was sent incorrectly
contact one of the administrators:
> > > > > > > > > > > >   http://issues.apache.org/jira/secure/Administrators.jspa
> > > > > > > > > > > > -
> > > > > > > > > > > > For more information on JIRA,
see:
> > > > > > > > > > > >   http://www.atlassian.com/software/jira
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > ---------------------------------------------------------------------
> > > > > > > > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > > > > > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > > > > > > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > ---------------------------------------------------------------------
> > > > > > > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > > > > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > > > > > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > ---------------------------------------------------------------------
> > > > > > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > > > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > > > > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > ---------------------------------------------------------------------
> > > > > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > > > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > ---------------------------------------------------------------------
> > > > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > > >
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message