commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nikolaus Hansen (JIRA)" <>
Subject [jira] [Commented] (MATH-867) CMAESOptimizer with bounds fits finely near lower bound and coarsely near upper bound.
Date Sat, 29 Sep 2012 15:32:07 GMT


Nikolaus Hansen commented on MATH-867:

However v2 makes the remaining modifications simpler.
The problem is that we don't know what are "the remaining modifications": 
I thought I knew... 
{quote}we never got beyond to the point were both "testFitAccuracyDependsOnBoundary" and "testConstrainedRosen"
the former, because inputSigma was not adapted appropriately, the latter because the test
against boundaries was not adapted appropriately. 

There are two different things:
1. Does the existence of constraints modify the search procedure is some way (i.e. CMAES must
"know" that it deals with boundaries)?
yes. Otherwise it will sample and evaluation points outside the boundaries. 
2. Alternately, is it possible to pass a modified objective function (in which the allowed
range of the original objective function has been mapped to the [-inf, +inf] interval) and
have CMAES behave the same (i.e. find the same solution)?
probably not the same, but possibly reasonably well, unless the boundary is mapped to (or
close to) inf, which is likely to lead to unexpected results, if the optimum is on (or close
to) the boundary. 

checking MultivariateFunctionPenaltyAdapter (which does not map the parameters, rather computes
a penalty), the answer is likely to be no, if the optimal solution happens to be on (or very
close to) the bound. 

regarding the documentation of inputSigma: I don't see in what sense the doc says that it
depends on the bounds.
Then what is "initial search volume"?
the volume/region where points are likely to be sampled in the beginning of the search. Points
beyond, say, startpoint + 10*sigma are not likely to be sampled in the beginning. 

I interpret the doc as roughly saying "0.3 times the range". Perhaps this is wrong, in which
case it should be made clearer...
I agree, it is not clear (I guess it was taken from another doc and slightly changed context).

We noticed that very small or very large values for "sigma" did not work; so maybe we should
say "inputSigma must be of order 1" .
no, it entirely depends on how far we expect the optimal solution to be from the start solution.
Putting it differently: one could rescale parameters by a factor 1000 (say in one case it
is km, in the other m), then one would need to rescale inputSigma accordingly (which would
not be in the order of one). 

Codes I write typically don't accept a default value for inputSigma, basically for this reason.
I agree however that a value of 1 might often turn out OK. 

> CMAESOptimizer with bounds fits finely near lower bound and coarsely near upper bound.

> ---------------------------------------------------------------------------------------
>                 Key: MATH-867
>                 URL:
>             Project: Commons Math
>          Issue Type: Bug
>            Reporter: Frank Hess
>         Attachments: MATH867_patch,
> When fitting with bounds, the CMAESOptimizer fits finely near the lower bound and coarsely
near the upper bound.  This is because it internally maps the fitted parameter range into
the interval [0,1].  The unit of least precision (ulp) between floating point numbers is much
smaller near zero than near one.  Thus, fits have much better resolution near the lower bound
(which is mapped to zero) than the upper bound (which is mapped to one).  I will attach a
example program to demonstrate.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message