Mailing-List: contact dev-help@commons.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Commons Developers List" <dev@commons.apache.org>
Received-SPF: pass (nike.apache.org: domain of phil.steitz@gmail.com
 designates 209.85.160.50 as permitted sender)
From: Phil Steitz <phil.steitz@gmail.com>
Content-Type: text/plain;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (1.0)
Subject: Re: [jira] [Commented] (MATH-1131) Kolmogorov-Smirnov Tests takes
 'forever' on 10,000 item dataset
Message-Id: <E94E3BF1-DD2B-479B-ACB0-556E4C8904C3@gmail.com>
Date: Wed, 25 Jun 2014 13:09:28 -0700
References: <JIRA.12723580.1403683968086@arcas>
 <JIRA.12723580.1403683968086.48286.1403720186235@arcas>
In-Reply-To: <JIRA.12723580.1403683968086.48286.1403720186235@arcas>
To: Commons Developers List <dev@commons.apache.org>

Sorry for responding to the list but I have only mobile atm .  IIRC the roun=
dedK method should not be creating matrices of BigFractions, but rather usin=
g doubles.=20

> On Jun 25, 2014, at 11:16 AM, "Thomas Neidhart (JIRA)" <jira@apache.org> w=
rote:
>=20
>=20
>    [ https://issues.apache.org/jira/browse/MATH-1131?page=3Dcom.atlassian.=
jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D140438=
57#comment-14043857 ]=20
>=20
> Thomas Neidhart commented on MATH-1131:
> ---------------------------------------
>=20
> I did briefly debug the example and indeed the calculation hangs when call=
ing roundedK, or more precisely in createH.
>=20
> There powers of BigFraction objects are created with really big numerators=
 and denominators. Some of the calculations later on take then forever becau=
se of this, e.g. when internally calculating the gcd.
>=20
> Looking at the implementation from the referenced paper, there the H value=
s are computed with double precision. Was there a specific reason to use Big=
Fraction in our implementation? Is there a specific need for that level of a=
ccuracy for the Kolmogorov-Smirnov Test? The other inference tests do not se=
em to be so stringent.
>=20
> It looks like there is no easy way to limit the maxDenominator when callin=
g multiply() as it is possible when creating a BigFraction object.
>=20
>=20
>> Kolmogorov-Smirnov Tests takes 'forever' on 10,000 item dataset
>> ---------------------------------------------------------------
>>=20
>>                Key: MATH-1131
>>                URL: https://issues.apache.org/jira/browse/MATH-1131
>>            Project: Commons Math
>>         Issue Type: Bug
>>   Affects Versions: 3.3
>>        Environment: Java 8
>>           Reporter: Schalk W. Cronj=C3=A9
>>        Attachments: 1.txt, ReproduceKsIssue.groovy, ReproduceKsIssue.java=

>>=20
>>=20
>> I have code simplified to the following:
>>    KolmogorovSmirnovTest kst =3D new KolmogorovSmirnovTest();
>>    NormalDistribution nd =3D new NormalDistribution(mean,stddev);
>>    kst.kolmogorovSmirnovTest(nd,dataset)
>> I find that for my dataset of 10,000 items, the call to kolmogorovSmirnov=
Test takes 'forever'. It has not returned after nearly 15minutes and in one m=
y my tests has gone over 150MB in  memory usage.
>=20
>=20
>=20
> --
> This message was sent by Atlassian JIRA
> (v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org