commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Phil Steitz <phil.ste...@gmail.com>
Subject Re: [math] SummaryStatistics.setVarianceImpl Usage
Date Sun, 16 Oct 2011 19:17:49 GMT
On 10/16/11 10:55 AM, Warren Tang wrote:
> I've manually calculated it (1.1180339887498948482045868343656).
> So Mikkel's is right.

Thanks for reporting this.  This points to another bug.  Ugh.  I
will open it.

Phil
>
> Regards,
> Warren Tang <http://blog.tangcs.com>
>
> On 10/17/2011 1:49 AM, Warren Tang wrote:
>> Hi, Phil
>>
>> Your workaround does not work. Switching the value of
>> isBiasCorrected between true and false produces the same standard
>> deviation (1.2909944487358056), which is different from the
>> result of Mikkel's formula (1.118033988749895).
>>
>> Regards,
>> Warren Tang <http://blog.tangcs.com>
>>
>> On 10/17/2011 1:20 AM, Phil Steitz wrote:
>>> On 10/16/11 8:24 AM, Mikkel Meyer Andersen wrote:
>>>> Dear Warren,
>>>>
>>>> This is probably a bug. Sorry for this. Would you be so kind to
>>>> report
>>>> it as described
>>>> onhttp://commons.apache.org/math/issue-tracking.html
>>>> .
>>>>
>>>> What you can do instead is this:
>>>> int[] scores = {1, 2, 3, 4};
>>>>
>>>> SummaryStatistics stats = new SummaryStatistics();
>>>> for(int i : scores) {
>>>>      stats.addValue(i);
>>>> }
>>>> double sd = FastMath.sqrt(stats.getSecondMoment() / stats.getN());
>>>>
>>>> System.out.println(sd);
>>>>
>>>> So, calculating sd as:
>>>> double sd = FastMath.sqrt(stats.getSecondMoment() / stats.getN());
>>>>
>>>> And then there is no need to stats.setVarianceImpl(new
>>>> Variance(false)).
>>> Yes, this is a bug.  Lets track it.  The problem is that the
>>> code is
>>> using instanceOf check to identify overridden impls, which fails in
>>> this case.  This needs to be fixed.
>>>
>>> Another workaround that should work is to get the default variance
>>> impl and set its biasCorrected property:
>>>
>>> SummaryStatistics stats = new SummaryStatistics();
>>> Variance variance = (Variance) stats.getVarianceImpl();
>>> variance.setBiasCorrected(false);
>>>
>>> then just use the stats instance directly and the reported variance
>>> should be non-bias-corrected.
>>>
>>> Phil
>>>> Cheers, Mikkel.
>>>>
>>>> 2011/10/16 Warren Tang<warren.c.tang@gmail.com>:
>>>>> Hi, Mikkel
>>>>>
>>>>> I'm using commons-math 2.2. The code to reproduce the issue.
>>>>>
>>>>> import
>>>>> org.apache.commons.math.stat.descriptive.SummaryStatistics;
>>>>> import org.apache.commons.math.stat.descriptive.moment.Variance;
>>>>>
>>>>>   @Test public void testStandardDeviation() {
>>>>>     int[] scores = {1, 2, 3, 4};
>>>>>     SummaryStatistics stats = new SummaryStatistics();
>>>>>     stats.setVarianceImpl(new Variance(false)); //use
>>>>> "population variance"
>>>>>     for(int i : scores) {
>>>>>     stats.addValue(i);
>>>>>     }
>>>>>     double sd = stats.getStandardDeviation();
>>>>>     System.out.println(sd);
>>>>>   }
>>>>>
>>>>> Regards,
>>>>> Warren Tang<http://blog.tangcs.com>
>>>>>
>>>>> On 10/16/2011 10:43 PM, Mikkel Meyer Andersen wrote:
>>>>>> Dear Warren,
>>>>>>
>>>>>> Could you provide values for the scores-variable in the current
>>>>>> example making it possible to reproduce?
>>>>>>
>>>>>> Are you in fact using version 1.2 as reflected by the link
>>>>>> you gave?
>>>>>> Or which version are you using?
>>>>>>
>>>>>> Cheers, Mikkel.
>>>>>>
>>>>>> 2011/10/16 Warren Tang<warren.c.tang@gmail.com>:
>>>>>>> Hello, everyone
>>>>>>>
>>>>>>> I'm trying to get a "population standard deviation
>>>>>>>
>>>>>>> <http://commons.apache.org/math/api-1.2/org/apache/commons/math/stat/descriptive/moment/StandardDeviation.html>"
>>>>>>>
>>>>>>> (non-bias-corrected) from SummaryStatistics.
>>>>>>>
>>>>>>> This is what I did:
>>>>>>>
>>>>>>> SummaryStatistics stats = new SummaryStatistics();
>>>>>>> stats.setVarianceImpl(new Variance(false)); //use
>>>>>>> "population variance"
>>>>>>> ( sum((x_i - mean)^2) / n )
>>>>>>> for(int i : scores) {
>>>>>>> stats.addValue(i);
>>>>>>> }
>>>>>>> double sd = stats.getStandardDeviation();
>>>>>>>
>>>>>>> However, the value of "sd" is "NaN". How can I do it correctly?
>>>>>>>
>>>>>>> -- 
>>>>>>> Regards,
>>>>>>> Warren Tang<http://blog.tangcs.com>
>>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>>
>>>>>> To unsubscribe, e-mail:user-unsubscribe@commons.apache.org
>>>>>> For additional commands, e-mail:user-help@commons.apache.org
>>>>>>
>>>> ---------------------------------------------------------------------
>>>>
>>>> To unsubscribe, e-mail:user-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail:user-help@commons.apache.org
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>>
>>> To unsubscribe, e-mail:user-unsubscribe@commons.apache.org
>>> For additional commands, e-mail:user-help@commons.apache.org
>>>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org


Mime
View raw message