commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Warren Tang <warren.c.t...@gmail.com>
Subject Re: [math] SummaryStatistics.setVarianceImpl Usage
Date Sun, 16 Oct 2011 17:55:41 GMT
I've manually calculated it (1.1180339887498948482045868343656). So 
Mikkel's is right.

Regards,
Warren Tang <http://blog.tangcs.com>

On 10/17/2011 1:49 AM, Warren Tang wrote:
> Hi, Phil
>
> Your workaround does not work. Switching the value of isBiasCorrected 
> between true and false produces the same standard deviation 
> (1.2909944487358056), which is different from the result of Mikkel's 
> formula (1.118033988749895).
>
> Regards,
> Warren Tang <http://blog.tangcs.com>
>
> On 10/17/2011 1:20 AM, Phil Steitz wrote:
>> On 10/16/11 8:24 AM, Mikkel Meyer Andersen wrote:
>>> Dear Warren,
>>>
>>> This is probably a bug. Sorry for this. Would you be so kind to report
>>> it as described onhttp://commons.apache.org/math/issue-tracking.html
>>> .
>>>
>>> What you can do instead is this:
>>> int[] scores = {1, 2, 3, 4};
>>>
>>> SummaryStatistics stats = new SummaryStatistics();
>>> for(int i : scores) {
>>>      stats.addValue(i);
>>> }
>>> double sd = FastMath.sqrt(stats.getSecondMoment() / stats.getN());
>>>
>>> System.out.println(sd);
>>>
>>> So, calculating sd as:
>>> double sd = FastMath.sqrt(stats.getSecondMoment() / stats.getN());
>>>
>>> And then there is no need to stats.setVarianceImpl(new Variance(false)).
>> Yes, this is a bug.  Lets track it.  The problem is that the code is
>> using instanceOf check to identify overridden impls, which fails in
>> this case.  This needs to be fixed.
>>
>> Another workaround that should work is to get the default variance
>> impl and set its biasCorrected property:
>>
>> SummaryStatistics stats = new SummaryStatistics();
>> Variance variance = (Variance) stats.getVarianceImpl();
>> variance.setBiasCorrected(false);
>>
>> then just use the stats instance directly and the reported variance
>> should be non-bias-corrected.
>>
>> Phil
>>> Cheers, Mikkel.
>>>
>>> 2011/10/16 Warren Tang<warren.c.tang@gmail.com>:
>>>> Hi, Mikkel
>>>>
>>>> I'm using commons-math 2.2. The code to reproduce the issue.
>>>>
>>>> import org.apache.commons.math.stat.descriptive.SummaryStatistics;
>>>> import org.apache.commons.math.stat.descriptive.moment.Variance;
>>>>
>>>>   @Test public void testStandardDeviation() {
>>>>     int[] scores = {1, 2, 3, 4};
>>>>     SummaryStatistics stats = new SummaryStatistics();
>>>>     stats.setVarianceImpl(new Variance(false)); //use "population variance"
>>>>     for(int i : scores) {
>>>>     stats.addValue(i);
>>>>     }
>>>>     double sd = stats.getStandardDeviation();
>>>>     System.out.println(sd);
>>>>   }
>>>>
>>>> Regards,
>>>> Warren Tang<http://blog.tangcs.com>
>>>>
>>>> On 10/16/2011 10:43 PM, Mikkel Meyer Andersen wrote:
>>>>> Dear Warren,
>>>>>
>>>>> Could you provide values for the scores-variable in the current
>>>>> example making it possible to reproduce?
>>>>>
>>>>> Are you in fact using version 1.2 as reflected by the link you gave?
>>>>> Or which version are you using?
>>>>>
>>>>> Cheers, Mikkel.
>>>>>
>>>>> 2011/10/16 Warren Tang<warren.c.tang@gmail.com>:
>>>>>> Hello, everyone
>>>>>>
>>>>>> I'm trying to get a "population standard deviation
>>>>>>
>>>>>> <http://commons.apache.org/math/api-1.2/org/apache/commons/math/stat/descriptive/moment/StandardDeviation.html>"
>>>>>> (non-bias-corrected) from SummaryStatistics.
>>>>>>
>>>>>> This is what I did:
>>>>>>
>>>>>> SummaryStatistics stats = new SummaryStatistics();
>>>>>> stats.setVarianceImpl(new Variance(false)); //use "population variance"
>>>>>> ( sum((x_i - mean)^2) / n )
>>>>>> for(int i : scores) {
>>>>>> stats.addValue(i);
>>>>>> }
>>>>>> double sd = stats.getStandardDeviation();
>>>>>>
>>>>>> However, the value of "sd" is "NaN". How can I do it correctly?
>>>>>>
>>>>>> --
>>>>>> Regards,
>>>>>> Warren Tang<http://blog.tangcs.com>
>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail:user-unsubscribe@commons.apache.org
>>>>> For additional commands, e-mail:user-help@commons.apache.org
>>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail:user-unsubscribe@commons.apache.org
>>> For additional commands, e-mail:user-help@commons.apache.org
>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:user-unsubscribe@commons.apache.org
>> For additional commands, e-mail:user-help@commons.apache.org
>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message