commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Neidhart (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MATH-1179) kolmogorovSmirnovTest poor performance in monteCarloP method
Date Fri, 01 May 2015 10:24:06 GMT

    [ https://issues.apache.org/jira/browse/MATH-1179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14523013#comment-14523013
] 

Thomas Neidhart commented on MATH-1179:
---------------------------------------

Regarding the inaccurate result for the approximateP method: this is due to the ksSum method
whose series expansion does not work well for d values < 1. Using the pelzGood method in
this case improves the result a lot.

Regarding the slowness: I think it would be better to adapt the logic to decide which method
is used in the kolmogorovSmirnovTest. As explained above, for sizes < 10000 the monte carlo
method is used, but it seems to be too slow for such large arrays. Either we reduce this number
considerably, or even drop the monte carlo method completely from there. A user can still
call it explicitly if needed/wanted. If we can improve the accuracy of the approximateP method,
this should be good enough imho?

> kolmogorovSmirnovTest poor performance in monteCarloP method
> ------------------------------------------------------------
>
>                 Key: MATH-1179
>                 URL: https://issues.apache.org/jira/browse/MATH-1179
>             Project: Commons Math
>          Issue Type: Bug
>            Reporter: Gilad
>             Fix For: 4.0
>
>         Attachments: KSTest-JavaAndR.txt, KSTestSnippet.txt
>
>
> I'm using the kolmogovSmirnovTest method to calculate pvalues.
> However, when i try running the test on two double[] of sizes 5 and 45 the results take
over 10 seconds to calculate.
> This seems very long, whereas in R it takes a few miliseconds for the same calculation.
> I'd be very happy to hear any comment you may have on the subject.
>    Gilad



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message