Mailing-List: contact dev-help@commons.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Commons Developers List" <dev@commons.apache.org>
Received-SPF: pass (athena.apache.org: local policy)
Message-ID: <521B9784.7090509@nrl.navy.mil>
Date: Mon, 26 Aug 2013 13:59:32 -0400
From: Evan Ward <evan.ward@nrl.navy.mil>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:17.0) Gecko/20130623 Thunderbird/17.0.7
MIME-Version: 1.0
To: <dev@commons.apache.org>
Subject: Re: [Math] Fluent API, inheritance and immutability
References: <e554b3c1e1ea16ade52b4aaf518cbe47@scarlet.be>
 <5202C477.7000805@nrl.navy.mil> <4bce809bf25424df26ac7c785e62e671@scarlet.be>
 <520350D8.8040806@spaceroots.org>
 <32c015fcad3340ed67a0a17dab5c2b82@scarlet.be> <5203A4FC.6090400@nrl.navy.mil>
 <06f3ad8c07d6ada845ba4f8937c90efa@scarlet.be> <5203DBE0.1070500@nrl.navy.mil>
 <52058C7C.6030105@gmail.com> <52090785.2090009@nrl.navy.mil>
 <52090B10.1040208@gmail.com> <5209219D.3000300@nrl.navy.mil>
 <e5d807d56eac4fa1c95fef3abfed6a49@scarlet.be> <520AA966.4070009@nrl.navy.mil>
 <fab9e7bf4fbec41be14006fa24d3ca1a@scarlet.be> <520BF62C.8000207@nrl.navy.mil>
 <7d9f072992c9493823f5d3b2a6c9adcf@scarlet.be>
 <CAOGo0VYmsQZe9UX08eh2_8p0a_4VFKysLje9fV04=t16wb=npA@mail.gmail.com>
 <9a6118265486240db4a2130d5119c8c1@scarlet.be>
 <CAMFBX4h0eo_k5hOEv__53s_Mr7ELseg3CjasTsi7EOZFtzRjMw@mail.gmail.com>
 <52136723.3020208@nrl.navy.mil>
 <CAMFBX4j1NdV4=4QYb-Fqo5N-yvduQKB=XXUzucxKHpj=PPgbiQ@mail.gmail.com>
In-Reply-To: 
 <CAMFBX4j1NdV4=4QYb-Fqo5N-yvduQKB=XXUzucxKHpj=PPgbiQ@mail.gmail.com>
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 7bit

Hi again,

I rearranged the least squares package and I've posted the results.[1]
I've also created a pull request[2] and an associated issue.[3]

[1] https://github.com/wardev/commons-math/tree/sepOpt
[2] https://github.com/apache/commons-math/pull/1
[3] https://issues.apache.org/jira/browse/MATH-1026

A summary of what I changed: (See the git log for more details.)

1. Separate LeastSquaresProblem from LeastSquaresOptimizer. Update tests
to use the new interfaces.
2. Immutable fluent builders for optimizers.
3. Remove weights from the LeastSquaresProblem interface. They are
pre-applied to the Jacobian and residuals.
4. Single function for computed values and Jacobian. I'm not 100% on the
name. Call it MultivariateVectorMatrixFunction? or
MultivariateJacobianFunction? or
MultivariateVectorAndMatrixFunctionWithANameLongerThan80Characters? :)
5. Make LevenbergMarquardt thread safe by using explicit temporary
parameters instead of instance fields.
6. In GaussNewton change useLU to an Enum
7. Change ConvergenceChecker<PointVectorValuePair> to
CovergenceChecker<Evaluation>. Would allow conditions like "RMS changes
less than 1e-3".
8. Use [math] linear package in interfaces (RealMatrix and RealVector
instead of double[] and double[][])

I did not understand some things in the test code:

1. Why is AbstractLeastSquaresOptimizerTestValidation (renamed to
EvaluationTestValidation) disabled by default and how do I tell if it
passed?
2. Other smaller questions about the tests. Theses are in comments in
the test package and marked with TODO.
3. LU and QR use a default singularity threshold. Caused test failures
when GaussNewton is configured to use QR. (MATH-1024)
4. A single test case in LevenburgMarquardt.testControlParameters() now
converges where it used to diverge. It estimates a reasonable rms after
15 iterations. I'm not sure why it used to diverge...

There are still several items on the TODO list. Code and discussion is
appreciated. :)

Affecting the interface:

4. Immutable fluent builders for LSProblem?
6. maxIterations and ConvergenceChecker seem to have duplicate
functionality. (To stop the algorithm after a certain number of
iterations.) Remove the maxIterations parameter and let the
ConvergenceCheck handle that job.

Implementation only:

2. Efficient diagonal weights. (don't need to create a new matrix, just
multiply in place)
3. Add more convenience methods to Factory and Builder. E.g. for
diagonal weights, ...
4. Add conditional logic to Factory so it can use efficient diagonal
weights even if they are provided in a matrix.
3. Use [math] linear package in optimizer implementations. To match the
current code in performance I think this would mean adding "views" or
specialized multiplication methods to the linear package. Depends on
MATH-765 and MATH-928.

Thanks for looking over the proposal and reviewing it.

Regards,
Evan

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org