commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gilles (JIRA)" <>
Subject [jira] Commented: (MATH-413) Miscellaneous issues concerning the "optimization" package
Date Tue, 31 Aug 2010 10:45:14 GMT


Gilles commented on MATH-413:

IMO, the most urgent issue to be dealt with is the design decision to make the convergence
checking criteria independent from the optimization algorithm. [This has a direct influence
on the API and should be stabilized in v3.0.]

Points 1, 2, 6, 10 in the issue description indicate the problems with this approach.

IMO, while modularization is often a good thing, the attempt produces several drawbacks in
this case:
* The user interface is more difficult than it could be: the user has to specify a "complex"
{{ConvergenceChecker}} object, instead of just {{double}}s (relative and/or absolute threshold/tolerance/accuracy).
* At the CM level, there is no single implementation of {{ConvergenceChecker}} that is applicable
for every algorithm (see {{BrentOptimizer}} and {{LevenbergMarquardtOptimizer}}).
* If we don't set a default, the user will get a NPE.
* If we set a default convergence checker, in an attempt to make usage easier, it will still
not be very useful because the user will have to set a new checker as soon as he needs to
change the algorithm accuracy.
* The user cannot rely on the description of convergence given by a {{ConvergenceChecker}}'s
implementation because if he chooses a checker that is "incompatible" with the optimization
algorithm, the behaviour is "undefined". [I have overridden {{setConvergenceChecker}} to forbid
the setting of anything other than a {{BrentConvergenceChecker}} but this completely voids
the intended flexibility of having a separate convergence checking procedure.]
* If we want to enforce the strict separation between optimization and convergence checking,
we'll have to _modify_ standard algorithm so that they fit into the mold (i.e. we must ensure
that "convergence checking" and "optimization" parts are indeed independent). This may not
even be possible without removing features of some algorithms (e.g. specific tolerance definitions
in "Levenberg-Marquardt"). And if it is done, then the resulting algorithm will not be standard

In light of all these problems, we should seriously re-examine whether the {{ConvergenceChecker}}
approach is the right way to go.

In the end, it is very well possible the best that can be done is to pass appropriate arguments
to the constructor of each optimizer's implementation. [With the consequence that if one wants
to change the convergence checking behaviour, one has to create a new instance. (Side note:
this is consistent with going towards instances' immutability.)]

> Miscellaneous issues concerning the "optimization" package
> ----------------------------------------------------------
>                 Key: MATH-413
>                 URL:
>             Project: Commons Math
>          Issue Type: Bug
>            Reporter: Gilles
>             Fix For: 3.0
> Revision 990792 contains changes triggered the following issues:
> * [MATH-394|]
> * [MATH-397|]
> * [MATH-404|]
> This issue collects the currently still unsatisfactory code (not necessarily sorted in
order of annoyance):
> # "BrentOptimizer": a specific convergence checker must be used. "LevenbergMarquardtOptimizer"
also has specific convergence checks.
> # Trying to make convergence checking independent of the optimization algorithm creates
problems (conceptual and practical):
>  ** See "BrentOptimizer" and "LevenbergMarquardtOptimizer", the algorithm passes "points"
to the convergence checker, but the actual meaning of the points can very well be different
in the caller (optimization algorithm) and the callee (convergence checker).
>  ** In "PowellOptimizer" the line search ("BrentOptimizer") tolerances depend on the
tolerances within the main algorithm. Since tolerances come with "ConvergenceChecker" and
so can be changed at any time, it is awkward to adapt the values within the line search optimizer
without exposing its internals ("BrentOptimizer" field) to the enclosing class ("PowellOptimizer").
> # Given the numerous changes, some Javadoc comments might be out-of-sync, although I
did try to update them all.
> # Class "DirectSearchOptimizer" (in package "") inherits from class
"AbstractScalarOptimizer" (in package "optimization.general").
> # Some interfaces are defined in package "optimization" but their base implementations
(abstract class that contain the boiler-plate code) are in package "optimization.general"
(e.g. "DifferentiableMultivariateVectorialOptimizer" and "BaseAbstractVectorialOptimizer").
> # No check is performed to ensure the the convergence checker has been set (see e.g.
"BrentOptimizer" and "PowellOptimizer"); if it hasn't there will be a NPE. The alternative
is to initialize a default checker that will never be used in case the user had intended to
explicitly sets the checker.
> # "NonLinearConjugateGradientOptimizer": Ugly workaround for the checked "ConvergenceException".
> # Everywhere, we trail the checked "FunctionEvaluationException" although it is never
> # There remains some duplicate code (such as the "multi-start loop" in the various "MultiStart..."
> # The "ConvergenceChecker" interface is very general (the "converged" method can take
any number of "...PointValuePair"). However there remains a "semantic" problem: One cannot
be sure that the list of points means the same thing for the caller of "converged" and within
the implementation of the "ConvergenceChecker" that was independently set.
> # It is not clear whether it is wise to aggregate the counter of gradient evaluations
to the function evaluation counter. In "LevenbergMarquartdOptimizer" for example, it would
be unfair to do so. Currently I had to remove all tests referring to gradient and Jacobian
> # In "AbstractLeastSquaresOptimizer" and "LevenbergMarquardtOptimizer", occurences of
"OptimizationException" were replaced by the unchecked "ConvergenceException" but in some
cases it might not be the most appropriate one.
> # "MultiStartUnivariateRealOptimizer": in the other classes ("MultiStartMultivariate...")
similar to this one, the randomization is on the firts-guess value while in this class, it
is on the search interval. I think that here also we should randomly choose the start value
(within the user-selected interval).
> # The Javadoc utility raises warnings (see output of "mvn site") which I couldn't figure
out how to correct.
> # Some previously existing classes and interfaces have become no more than a specialisation
of new "generics" classes; it might be interesting to remove them in order to reduce the number
of classes and thus limit the potential for confusion.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message