commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Otmar Ertl (JIRA)" <>
Subject [jira] [Commented] (MATH-1154) Statistical tests in stat.inference package are very slow due to implicit RandomGenerator initialization
Date Wed, 08 Oct 2014 11:03:34 GMT


Otmar Ertl commented on MATH-1154:

I agree with Phil that my proposed patch does not address the core problem, which is definitely
MATH-1124. The patch was thought as a short-term fix, which improves the performance essentially
with minimal invasive code changes. Once the root issue is solved, both patches proposed here
(mine and Thomas') will become obsolete anyway. The question is when will MATH-1124 be resolved?
In the case it is fixed for the next release, none of both patches need to be applied. If
not, it makes sense to apply one of both patches for the short-term. The question remains,
do you only want a speed up of the statistical tests (Thomas' patch) or also of the distribution
classes (my patch) ? 

> Statistical tests in stat.inference package are very slow due to implicit RandomGenerator
> --------------------------------------------------------------------------------------------------------
>                 Key: MATH-1154
>                 URL:
>             Project: Commons Math
>          Issue Type: Bug
>    Affects Versions: 3.3
>            Reporter: Otmar Ertl
>         Attachments: MATH-1154.patch, math3.patch
> Some statistical tests defined in the stat.inference package (e.g. BinomialTest or ChiSquareTest)
are unnecessarily very slow (up to a factor 20 slower than necessary). The reason is the implicit
slow initialization of a default (Well19937c) random generator instance each time a test is
performed. The affected tests create some distribution instance in order to use some methods
defined therein. However, they do not use any method for random generation. Nevertheless a
random number generator instance is automatically created when creating a distribution instance,
which is the reason for the serious slowdown. The problem is related to MATH-1124.
> There are following solutions:
> 1) Fix the affected statistical tests by passing a light-weight RandomGenerator implementation
(or even null) to the constructor of the distribution.
> 2) Or use for all distributions a RandomGenerator implementation that uses lazy initialization
to generate the Well19937c instance as late as possible. This would also solve MATH-1124.
> I will attach a patch proposal together with a performance test, that will demonstrate
the speed up after a fix.

This message was sent by Atlassian JIRA

View raw message