Mailing-List: contact dev-help@commons.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Commons Developers List" <dev@commons.apache.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: 7bit
Date: Mon, 21 Sep 2015 23:15:11 +0200
From: Gilles <gilles@harfang.homelinux.org>
To: <dev@commons.apache.org>
Subject: Re: [Math] LeastSquaresOptimizer Design
In-Reply-To: <55FF1138.1010003@gmail.com>
References: <55FE4E61.3020203@gmail.com>
 <0fb697bf08561e0b0d1cdf9a74cb36e4@scarlet.be> <55FF1138.1010003@gmail.com>
Message-ID: <211c61392f6933ede2ae596b03f53db6@scarlet.be>
User-Agent: Scarlet Webmail
X-DCC-scarlet.be-Metrics: eir; whitelist
X-Virus-Scanned: clamav-milter 0.98.1-exp at eir
X-Virus-Status: Clean

Hi.

On Sun, 20 Sep 2015 15:04:08 -0500, Ole Ersoy wrote:
> On 09/20/2015 05:51 AM, Gilles wrote:
>> On Sun, 20 Sep 2015 01:12:49 -0500, Ole Ersoy wrote:
>>> Wanted to float some ideas for the LeastSquaresOptimizer (Possibly
>>> General Optimizer) design.  For example with the
>>> LevenbergMarquardtOptimizer we would do:
>>> `LevenbergMarquardtOptimizer.optimize(OptimizationContext c);`
>>>
>>> Rough optimize() outline:
>>> public static void optimise() {
>>> //perform the optimization
>>> //If successful
>>>     c.notify(LevenberMarquardtResultsEnum.SUCCESS, solution);
>>> //If not successful
>>>
>>> 
>>> c.notify(LevenberMarquardtResultsEnum.TOO_SMALL_COST_RELATIVE_TOLERANCE,
>>> diagnostic);
>>> //or
>>>
>>> 
>>> c.notify(LevenberMarquardtResultsEnum.TOO_SMALL_PARAMETERS_RELATIVE_TOLERANCE,
>>> diagnostic)
>>> //etc
>>> }
>>>
>>> The diagnostic, when turned on, will contain a trace of the last N
>>> iterations leading up to the failure.  When turned off, the 
>>> Diagnostic
>>> instance only contains the parameters used to detect failure. The
>>> diagnostic could be viewed as an indirect way to log optimizer
>>> iterations.
>>>
>>> WDYT?
>>
>> I'm wary of having several different ways to convey information to 
>> the
>> caller.
> It would just be one way.

One way for optimizer, one way for solvers, one way for ...

> But the caller may not be the receiver
> (It could be).  The receiver would be an observer attached to the
> OptimizationContext that implements an interface allowing it to 
> observe
> the optimization.

I'm afraid that it will add to the questions of what to put in the
code and how.  [We already had sometimes heated discussions just for
the IMHO obvious (e.g. code formatting, documentation, exception...).]

>> It seems that the reporting interfaces could quickly overwhelm
>> the "actual" code (one type of context per algorithm).
> There would one type of Observer interface per algorithm.  It would
> act on the solution and what are currently exceptions, although these
> would be translated into enums.

Unless I'm mistaken, the most common use-case for codes implemented
in a library such as CM is to provide a correct answer or bail out
in a non-equivocal way.

It would make the code more involved to handle a minority of
(undefined) cases. [Actual examples would be welcome in order to
focus the discussion.]

>> The current reporting is based on exceptions, and assumes that if no
>> exception was thrown, then the user's request completed 
>> successfully.
> Sure - personally I'd much rather deal with something similar to an
> HTTP status code in a callback, than an exception .  I think the code
> is cleaner and the calback makes it more elegant to apply an adaptive
> approach to handling the response, like slightly relaxing 
> constraints,
> convergence parameters, etc.  Also by getting rid of the exceptions,
> we no longer depend on the I18N layer that they are tied to and now
> the messages can be more informative, since they target the root
> cause.  The observer can also run in the 'main' thread' while the
> optimization can run asynchronously.  Also WRT JDK9 and modules,
> loosing the exceptions would mean one less dependency when the 
> library
> is up into JDK9 modules...which would be more in line with this
> philosophy:
> https://github.com/substack/browserify-handbook#module-philosophy

I'm not sure I fully understood the philosophy from the text in this
short paragraph.
But I do not agree with the idea that the possibility to quickly find
some code is more important than standards and best practices.

>> I totally agree that in some circumstances, more information on the
>> inner working of an algorithm would be quite useful.
> ... Algorithm iterations become unit testable.
>>
>> But I don't see the point in devoting resources to reinvent the 
>> wheel:
> You mean pimping the wheel?  Big pimpin.

I think that logging statements are easy to add, not disruptive at all,
and come in handy to understand a code's unexpected behaviour.
Assuming that a "logging" feature is useful, it can be added *now* 
using
a dependency towards a weight-less (!) framework such as "slf4j".
IMO, it would be a waste of time to implement a new communication layer
that can do that, and more, if it would be used for logging only in 99%
of the cases.

>>
>> I longed several times for the use of a logging library.
>> The only show-stopper has been the informal "no-dependency" 
>> policy...
> JDK9 Jigsaw should solve dependency hell, so the less coupling
> between commons math classes the better.

I wouldn't call "coupling" the dependency towards exception classes:
they are little utilities that can make sense in various parts of the
library.

[Unless one wants to embark on yet another discussion about exceptions;
whether there should be one class for each of the "messages" that exist
in "LocalizedFormats"; whether localization should be done in CM;
etc.]

> Anyways I'm obviously
> interested in playing with this stuff, so when I get something up 
> into
> a repository I'll to do a callback :).

If you are interested in big overhauls, there is one that gathered
relative consensus: rewrite the algorithms in a "multithread-friendly"
way.
Some ideas were floated (cf. ML archive) but no implementation or
experiment...  Perhaps with a well-defined goal such as performance
improvement, your design suggestions will become clearer to more 
people.

AFAIK, only the classes in the "o.a.c.m.neuralnet" package are 
currently
ready to be used with the "java.util.concurrent" framework.


Best regards,
Gilles

>
> Cheers,
> Ole
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org