commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Luc Maisonobe <>
Subject Re: [math] redesigning the optimization/estimation packages
Date Wed, 14 Jan 2009 20:08:28 GMT
Gilles Sadowski a écrit :
> Hi.
>> The first step would be to rearrange slightly the analysis package. We
>> could have the main interfaces in the analysis package
>> (UnivariateRealFunction, DifferentiableUnivariateRealFunction ...) and
>> spread the other interfaces and classes in several subpackages:
>>  - analysis.solving
>>  - analysis.integration
>>  - analysis.interpolation
>> The second step would be to add an analysis.minimization package. The
>> contributed Brent minimization would get there, probably with a simple
>> gold section minimizer too.
> Could we have "root" instead of "solving"?
> Also we could have "minimum" instead of "minimization".
> I think that "root" is clearer (e.g. for a new user browsing the package
> list javadoc).
> [And both are little bit shorter names than the alternative. ;-)]

I am OK with any suggestion like this. Since I'm not a native english
speaker, I let some time for other people to express their feelings
about this.

>> I think a dedicated interface
>> UnivariateRealMinimizer would be needed instead of reusing the
>> UnivariateRealSolver one, both interface have different goals. They
>> probably share many methods but not the underlying semantics.
> I'd rather re-use as much code as possible.
> Which methods do not have the same semantics?  The "getResult" method, for
> example, can return a root location as well as a minimum location.

Yes, but a root is not a minimum location. For me, this is a sufficient
reason to separate interfaces. From a user point of view, knowing that
SolverXyz implements UnivariateRealSolver is sufficient to understand
what solve mean and what are the properties of the returned value. They
would not be the same (even if we reuse the name "solve") if the class
implement UnivariateRealMinimizer.

This is especially true for Brent solver. Two algorithms exist with the
same name, one for root one for minimum. I guess some users may get
confused if we would link too tightly the two interfaces. They could mix
things up and compute a root when they really want to compute a minimum
or the other way round.

I agree with code reuse, though. Perhaps we could have a very generic
superinterface for the shared parts like convergence settings or max
iterations and things like that. But the "solve" methods should be
separate, and even probably have different names to stay on the safe side.

> What about creating new interfaces and classes in the future "minimization"
> (or "minimum") package that would extend those in "solving" (or "root")?
> public interface UnivariateRealMinimizer extends UnivariateRealSolver {
>   public double getValueAtMinimum();
> }
> public class UnivariateRealMinimizerImpl extends UnivariateRealSolverImpl
>   implements UnivariateRealMinimizer {
>   private double valueAtMinimum;
>   protected void setResult(double x,
>                            double fx,
>                            double iterationCount) {
>     setResult(x, iterationCount);
>     valueAtMinimum = fx;
>   }
>   public double getValueAtMinimum() {
>     return valueAtMinimum;
>   }
> }

I would prefer something like:

 public interface ConvergenceAlgorithm {
    void setMaximalIterationCount(int count);

 public interface UnivariateRealSolver extends ConvergenceAlgorithm {
    double solve(UnivariateRealFunction f, ...);

 public interface UnivariateRealMinimizer extends ConvergenceAlgorithm {
    double findMinimum(UnivariateRealFunction f, ...);

>> The third step would be to handle multivariate or multivalued functions.
>> Should separate parallel packages be created or not ?
>>>From a practical point of view, I'd think so: I'd guess that it is more
> common to use the univariate algorithms, and so, it would be convenient not
> to clutter those packages with classes that are useful only in the
> multi-dimensional case.
> Also, from what you say below, it seems that it will be much more work to
> refactor the framework for the multi-dimensional case, whereas the packages
> in the one-dimensional case could be "stabilized" sooner.

You are right, but we cannot postpone major change after 2.0. Once 2.0
will be out, we will try as much as possible to preserve compatibility
for users, so later major change would have to wait for 3.0. Now is
proper time to revamp everything we need.

I expect to have almost one month off work next month due to forced rest
after minor surgery operation. I will have plenty of time to kill then.

>> The minimization
>> part for these kind of functions would come from the low level classes
>> of the estimation package and from the optimization package. The former
>> ones correspond to algorithms using derivatives and the latter
>> correspond to algorithms using only function values. The interfaces
>> should be simplified to remain low-level and deal directly with function
>> values, target, residuals.
>> A fourth step would be to build higher level abstractions on top of the
>> previous ones, providing the concepts of measurements, parameters  and
>> linear/non-linear model ... These are mainly the existing
>> WeightedMeasurement, EstimatedParameter and EstimationProblem classes
>> and interfaces.
> We already use several librairies: "jMinuit" in Java and "Opt++" in C++
> (using JNI). Surely I'd prefer everything to be in Commons-Math...

Our current optimization part is still fairly basic, so for now it would
be safe to stay with those libraries. What is implemented does work and
works well, but there are not many algorithms and the API organization
is really awkward for now. Improving the organization to have a stable
framework is a goal for 2.0. Adding new algorithms and features to
become a decent alternative to these libraries is a goal for 2.x, except
if new motivated contributors step in ...


> Best,
> Gilles
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message