commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Schalk W. Cronjé (JIRA) <j...@apache.org>
Subject [jira] [Comment Edited] (MATH-1131) Kolmogorov-Smirnov Tests takes 'forever' on 10,000 item dataset
Date Wed, 25 Jun 2014 20:45:26 GMT

    [ https://issues.apache.org/jira/browse/MATH-1131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14044019#comment-14044019
] 

Schalk W. Cronjé edited comment on MATH-1131 at 6/25/14 8:44 PM:
-----------------------------------------------------------------

[~phil@steitz.com] said on the ML:

bq. Sorry for responding to the list but I have only mobile atm .  IIRC the roundedK method
should not be creating matrices of BigFractions, but rather using doubles.

I did a quick hack on the test code I used for createH earlier to use double instead and the
speed improvement as expected is immense - down from 36min to 9min. I cannot comment on whether
the change in precision is significant, but not was not the point of the test.




was (Author: ysb33r):
[~phil@steitz.com]said on the ML:

bq. Sorry for responding to the list but I have only mobile atm .  IIRC the roundedK method
should not be creating matrices of BigFractions, but rather using doubles.

I did a quick hack on the test code I used for createH earlier to use double instead and the
speed improvement as expected is immense - down from 36min to 9min. I cannot comment on whether
the change in precision is significant, but not was not the point of the test.



> Kolmogorov-Smirnov Tests takes 'forever' on 10,000 item dataset
> ---------------------------------------------------------------
>
>                 Key: MATH-1131
>                 URL: https://issues.apache.org/jira/browse/MATH-1131
>             Project: Commons Math
>          Issue Type: Bug
>    Affects Versions: 3.3
>         Environment: Java 8
>            Reporter: Schalk W. Cronjé
>         Attachments: 1.txt, ReproduceKsIssue.groovy, ReproduceKsIssue.java
>
>
> I have code simplified to the following:
>     KolmogorovSmirnovTest kst = new KolmogorovSmirnovTest();
>     NormalDistribution nd = new NormalDistribution(mean,stddev);
>     kst.kolmogorovSmirnovTest(nd,dataset)
> I find that for my dataset of 10,000 items, the call to kolmogorovSmirnovTest takes 'forever'.
It has not returned after nearly 15minutes and in one my my tests has gone over 150MB in 
memory usage. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message