accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mike Drob (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-2494) Stat calculation of STDEV may be inaccurate
Date Wed, 19 Mar 2014 20:35:44 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13940948#comment-13940948
] 

Mike Drob commented on ACCUMULO-2494:
-------------------------------------

I should have been more clear.

{quote}
I have seen multiple times in the past where small seemingly innocuous changes for minor bugs
have introduced critical bugs.  In this case TabletServer uses the Stat class, but does not
use the std deviation.  The risk is a small possibility of introducing a new critical bug
in tserver if the change breaks Stat in some strange new way.  The benefit of the change is
that informational output from a few test may be better.
{quote}
Based on this comment, I do not understand which version you would support including the new
implementation of Stat in. I also do not understand if you would support fixing stdev in older
versions while leaving everything else the same. FWIW, my implementation and the commons-math
implementation were the same (theirs was a bit more general and better documented, but otherwise
identical).

The reason I asked for clarification is that I cannot tell if you are -1 or -0 on including
the full change in 1.5.x.

{quote}
Something else that would cause problems would be if the Apache library stored all of the
data you were computing stats for.  I assume it does not do this, but would have to inspect
the code to be sure.
{quote}
It does not store the data, so this is not an issue.

> Stat calculation of STDEV may be inaccurate
> -------------------------------------------
>
>                 Key: ACCUMULO-2494
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-2494
>             Project: Accumulo
>          Issue Type: Bug
>          Components: client
>            Reporter: Mike Drob
>
> The math is sound, but it is susceptible to rounding errors. We should address that.
> See http://www.strchr.com/standard_deviation_in_one_pass and http://www.cs.berkeley.edu/~mhoemmen/cs194/Tutorials/variance.pdf



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message