commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sébastien Brisard (Commented) (JIRA) <j...@apache.org>
Subject [jira] [Commented] (MATH-699) inverseCumulativeDistribution fails with cumulative distribution having a plateau
Date Mon, 07 Nov 2011 04:04:51 GMT

    [ https://issues.apache.org/jira/browse/MATH-699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145211#comment-13145211
] 

Sébastien Brisard commented on MATH-699:
----------------------------------------

Hi,
I thought I would keep you up to date with the problems I'm facing at the moment.

First of all, I see no smart way to check that for sure, {{inverseCumulativeProbability(0)}}
(resp. {{inverseCumulativeProbability(1)}}) should return {{Double.NEGATIVE_INFINITY}} (resp.
{{Double.POSITIVE_INFINITY}}). What could be done would be to test for a neighbouring value,
and check that the cumulative probability is slightly above zero (resp. below one). But this
makes no sense, because the only neighbouring value which would make sense would be {{+/-Double.MAX_VALUE}},
and this surely return 0.0 or 1.0. So this is my first problem.

The way I see things is as follows: users might have no clue about the inverse cumulative
probability, but they *must* know the values of this inverse for p = 0 and p = 1. So I would
suggest to change the contract of {{getInitialDomain(p)}}. I would make clear in the javadoc
that {{getInitialDomain(p)}} should return {{Double.NEGATIVE_INFINITY}} *if, and only if*
{{inverseCumulativeProbability(0) == Double.NEGATIVE_INFINITY}}. Similarly, {{getInitialDomain(p)}}
should return {{Double.POSITIVE_INFINITY}} *if, and only if* {{inverseCumulativeProbability(1)
== Double.POSITIVE_INFINITY}}.

My second problem is to check for the presence of plateaux, consistently with finite precision.
Here is my initial idea. I first define the two absolute accuracies
* {{dx = getSolverAbsoluteAccuracy()}},
* {{dp = getSolverFunctionValueAccuracy()}}.

Then, if {{x}} is the root found by the solver: we do have {{cumulativeProbability( x ) ==
p}} (within a specified accuracy). The problem is to check whether there is a *smaller* value
which also satisfies this requirement (in which case, the smallest such value must be returned).

My initial test was {{cumulativeProbability(x - dx) == p}}. Then, I tried {{FastMath.abs(cumulativeProbability(x
- dx) - p) <= dp}}. Although more consistent with finite precision, this is not fully satisfactory,
because in simple cases (where there is no plateau), it might lead to the solver moving to
a somewhat less good point. At the very least, it would lead to additional iterations... to
finally get back to the initial point {{x}}.

Then, I thought of checking for *exact* nullity of the pdf at x. The problem is that the pdf
might have discontinuities, in which case, this simple test might fail (if {{x}} turns out
to be the higher-end of the plateau, and there is a slope discontinuity here).

So, I'm now back to this test: {{cumulativeProbability(x - dx) == cumulativeProbability( x
)}}. Please note
* *exact* equality test,
* I'm no longer testing for equality with the target value {{p}}, but with its estimate {{cumulativeProbability(
x )}}.

I would be grateful for some feedback on these issues. Also, I think it is clear that
* my code will require careful reviewers!!!
* it won't be completely fool-proof.
                
> inverseCumulativeDistribution fails with cumulative distribution having a plateau
> ---------------------------------------------------------------------------------
>
>                 Key: MATH-699
>                 URL: https://issues.apache.org/jira/browse/MATH-699
>             Project: Commons Math
>          Issue Type: Bug
>    Affects Versions: 3.0
>            Reporter: Sébastien Brisard
>            Assignee: Sébastien Brisard
>            Priority: Minor
>         Attachments: AbstractContinuousDistributionTest.java
>
>
> This bug report follows MATH-692. The attached unit test fails. As required by the definition
in MATH-692, the lower-bound of the interval on which the cdf is constant should be returned.
This is not so at the moment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

Mime
View raw message