Mailing-List: contact dev-help@commons.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Commons Developers List" <dev@commons.apache.org>
Received-SPF: pass (athena.apache.org: domain of
 SRS0=bIm0=KW=m4x.org=sebastien.brisard@bounces.m4x.org designates
 129.104.30.34 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <50DDDDAA.2030106@free.fr>
References: <50DDA635.8080908@spaceroots.org>
	<alpine.DEB.2.02.1212281553550.2532@astropc8.ulb.ac.be>
	<50DDBF6B.4040101@spaceroots.org>
	<alpine.DEB.2.02.1212281708000.2532@astropc8.ulb.ac.be>
	<50DDC860.7060703@gmail.com>
	<07E3B3D2-E7AE-4BAB-8872-ED88823B6200@gmail.com>
	<50DDDDAA.2030106@free.fr>
Date: Fri, 28 Dec 2012 21:18:56 +0100
Message-ID: 
 <CAGRH7HpruFv-2h1oGk7M091KM_VC2F6OiQYZ7dNhD4Y5=yhnNA@mail.gmail.com>
Subject: Re: [math] major problem with new released version 3.1
From: =?ISO-8859-1?Q?S=E9bastien_Brisard?= <sebastien.brisard@m4x.org>
To: Commons Developers List <dev@commons.apache.org>
Content-Type: multipart/alternative; boundary=047d7b6d9624808da904d1ef5ff6

--047d7b6d9624808da904d1ef5ff6
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hello,


2012/12/28 Luc Maisonobe <Luc.Maisonobe@free.fr>

> Le 28/12/2012 17:51, Konstantin Berlin a =E9crit :
> > Hi,
> >
> > I can understand Dimitri's frustration, it seems the optimization
> > framework gets worse with every iteration. However, we should
> > probably look forward and think about how to design it properly
> > instead.
> >
> > Several times I brought out some problems and ideas about the package
> > and it seems the only person who has an opinion is Gilles.
>
> Several people contributed to the thread (see
> <http://commons.markmail.org/thread/i6klmc2ytflb6qnt>), but as Gilles
> pointed out in one of the message, we lack an optimization expert. I
> sincerely would not want my opinion to be taken too seriously on this,
> so I only expressed what I could and did not decide anything by myself
> (only proposed to remove the wrong binding with DerivativeStructure,
> which has since been done).
>
> >
> > I will list what I consider to be major problems
>
> > 1) The OO design is bad, too much inheritance which could be handled
> > better by interfaces, the structure has no relation to the actual way
> > parts of optimizers can be mixed and matched. Input functions should
> > also have clear inheritance structure and should return both the
> > value, gradient, hessian in one function call.
>
> I strongly agree with Konstantin here. Abstract classes allow to add
> methods without breaking compatibility (which is good), but they also
> have some drawbacks (we have seen one drawback with the parameterized
> classes a few months ago, and this huge hierarchy is another one). So
> there is no silver bullet and we keep trying to find the good balance.
> As far as I am concerned, I would prefer we get fewer abstract classes,
> we remove some intermediate level (I don't know which ones), and we use
> more delegation than inheritance.
>
> > 2) Incorrect handling of constraints. There are only something like 5
> > possible constraints possible in optimization, with each
> > implementation of the solver handling some but not all. There is no
> > need to this runtime approach, which creates incredible amount of
> > confusion. All the possible constraints should be explicitly given in
> > the parameters to a function call, there are only 5. In addition,
> > constraints should be pre-processed a priori! So they should be an
> > input to the constructor not to the optimization function call.
>
> Our implementation for constraints is really limited (once again, scarce
> resources). What are the 5 types you consider? Simple/double bounds on
> parameters, linear/non-linear bounds and equality?
>
> > 3) Linear algebra package should be used as an interface and
> > internally to increase performance for larger datasets. Data copying
> > should be avoided as much as possible.
>
> Yes, but this would require solving another sub-problem first: having a
> decent sparse linear algebra implementation, which we also lack.


That's what I was going to write. At the moment, the current implementation
for sparse matrices and vector is deprecated, for lack of convincing fixes
for the bugs which have been found. These bugs are mainly related to the
fact that zero --the unstored value-- is signed, and the sign is lost in
sparse implementations. This might be considered as unimportant, but leads
to the fact that wa cannot achieve the same level of correctness in our
sparse impl as in our dense impls.

As for data copying, I strongly agree, and it is not really related to the
implementation of sparse vectors/matrices. For example, our implementation
of the conjugate gradient only requires a linear operator --not a matrix--,
which can be as sparse as you want. The problem is that all implementations
of iterative linear solvers use basic vector operations (add, sub, etc...),
which cannot be carried out in-place in the current abstract class
RealVector. This leads to unnecessary memory allocations for very large
problems. I am currently facing this issue. My matrix is not a problem
(it's actually a so-called "matrix-free" problem), and the vectors are
dense. The only problem is the demand in RAM. We should really give a
thought to in-place linear operations, without cluttering too much the API!
A possible way would be to use visitors.

Best regards,
S=E9bastien

Our
> implementation for full matrices was also in a sorry state prior to 3.0,
> but now fortunately this has improved at least for systems up to a few
> thousands rows and columns (so at least we do make progress on some
> points).
>
> > 4) Testing should be done on larger problems.
>
> Yes. I guess there are some general well known problems for that, so we
> should get a few of them and implement them. We did implement a number
> of tests from Minpack, but they focused on difficult cases rather than
> on problem size. I think optimization has a good testing coverage, but
> clearly large size problems is a needed addition.
>
> >
> > I know the response is that I am free to go implemented, but I think
> > we should at least agree on design principles instead of pushing
> > through your own ideas because the other person is too busy. The only
> > discussion we ever had on this was between me and Gilles, everyone
> > else disappeared.
>
> Well, we tried to keep up as our skills allowed, and we were also
> concerned with 3.1 being released at the same time.
>
> We have more time now than we had a few weeks ago. This is an
> opportunity to restart the discussion. We can refrain from pushing a new
> release (despite I would like this bug fix to be released officially)
> and take some time to think calmly. We could also push 3.2 with only the
> fix and without any revamp and start thinking about 4.0 with a redesign
> of these two main area: optimization and sparse linear algebra.
>
> If you could contribute to this discussion understanding we are not
> experts of this field and we cannot do it by ourselves, it would be great=
.
>
> best regards,
> Luc
>
> >
> > Thanks, Konstantin
> >
> > On Dec 28, 2012, at 11:27 AM, Phil Steitz <phil.steitz@gmail.com>
> > wrote:
> >
> >> On 12/28/12 8:12 AM, Dimitri Pourbaix wrote:
> >>> Luc,
> >>>
> >>>> So in order to make sure I understand your point, you would be
> >>>> OK if I deprecate the non-diagonal weights, in which case users
> >>>> needing this would have to implement it themselves by
> >>>> premultiplication (as both you and Konstantin seem to
> >>>> propose)?
> >>>
> >>> Yes, exactly.
> >>>
> >>>> Sure, but for the record the feature was also a last minute
> >>>> change. This was discussed on the list, and the final decision
> >>>> was to add this feature despite the release was close. No
> >>>> wonder we failed to test it thoroughsly.
> >>>
> >>> Last minute?  I have been discussing this with Gilles for
> >>> several months.
> >>
> >> Relevant project discussion happens *on this list*
> >>>
> >>>> We don't expect our releases to be perfect. We do our best,
> >>>> with the resources we have.
> >>>
> >>> I perfectly understand this but focusing those resources less on
> >>> rules and more on real cases might help.
> >>
> >> As stated before, you are more than welcome to *become* one of
> >> these resources.
> >>
> >> Phil
> >>>
> >>> Regards, Dim.
> >>>
> -------------------------------------------------------------------------=
---
> >>>
> >>>
> >>>
> Dimitri Pourbaix                         *      Don't worry, be happy
> >>> Institut d'Astronomie et d'Astrophysique *         and CARPE
> >>> DIEM. CP 226, office 2.N4.211, building NO     * Universite Libre
> >>> de Bruxelles            *      Tel : +32-2-650.35.71 Boulevard du
> >>> Triomphe                    *      Fax : +32-2-650.42.26 B-1050
> >>> Bruxelles                        *        NAC: HBZSC RG2Z6
> >>> http://sb9.astro.ulb.ac.be/~pourbaix     *
> >>> mailto:pourbaix@astro.ulb.ac.be
> >>>
> >>> ---------------------------------------------------------------------
> >>>
> >>>
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >>> For additional commands, e-mail: dev-help@commons.apache.org
> >>>
> >>>
> >>
> >>
> >> ---------------------------------------------------------------------
> >>
> >>
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >> For additional commands, e-mail: dev-help@commons.apache.org
> >>
> >
> >
> > ---------------------------------------------------------------------
> >
> >
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>

--047d7b6d9624808da904d1ef5ff6--