commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gilles (JIRA)" <>
Subject [jira] [Commented] (MATH-1154) Statistical tests in stat.inference package are very slow due to implicit RandomGenerator initialization
Date Tue, 07 Oct 2014 15:03:33 GMT


Gilles commented on MATH-1154:

bq. why do you constantly ignore me, and oppose everything I do or say?

Aren't you reversing the situation?

Apart from your patch with "null" RNGs, the other considerations (about MATH-1224, about improving
WellXXX RNGs) are related neither to the issue nor to the feature proposed to solve it.
It is (IMHO) clearer to separate concerns:
# Is the lazily initialized RNG a useful feature? \[Should be another issue (feature request).\]
# Is this issue (about tests) to be solved by initializing the RNG with "null"? \[A possible
fix for this issue.\]
# Is the default RNG (in a distribution instance) going to be a lazily-initialized RNG implementation?
\[Another possible fix for this issue (if and once the feature exists in CM).\]
# Is the default concrete RNG instance to be changed (from WellXxx to ...)? \[Unrelated.\]
# Is the implementation of WellXxx to be improved? \[Not directly related.\]
# Is the implementation of other RNG (which use insecure constructs) to be improved? \[My
addition of yet another similarly unrelated issue.\]

I'm proposing to tidy up things and process them in order, and you keep ignoring the suggestion.
If you and the OP want to resolve _this_ issue, then there is only one alternative at present:
Your fix. Please apply it and let's move on to the other suggestions.

> Statistical tests in stat.inference package are very slow due to implicit RandomGenerator
> --------------------------------------------------------------------------------------------------------
>                 Key: MATH-1154
>                 URL:
>             Project: Commons Math
>          Issue Type: Bug
>    Affects Versions: 3.3
>            Reporter: Otmar Ertl
>         Attachments: MATH-1154.patch, math3.patch
> Some statistical tests defined in the stat.inference package (e.g. BinomialTest or ChiSquareTest)
are unnecessarily very slow (up to a factor 20 slower than necessary). The reason is the implicit
slow initialization of a default (Well19937c) random generator instance each time a test is
performed. The affected tests create some distribution instance in order to use some methods
defined therein. However, they do not use any method for random generation. Nevertheless a
random number generator instance is automatically created when creating a distribution instance,
which is the reason for the serious slowdown. The problem is related to MATH-1124.
> There are following solutions:
> 1) Fix the affected statistical tests by passing a light-weight RandomGenerator implementation
(or even null) to the constructor of the distribution.
> 2) Or use for all distributions a RandomGenerator implementation that uses lazy initialization
to generate the Well19937c instance as late as possible. This would also solve MATH-1124.
> I will attach a patch proposal together with a performance test, that will demonstrate
the speed up after a fix.

This message was sent by Atlassian JIRA

View raw message