Mailing-List: contact dev-help@commons.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Commons Developers List" <dev@commons.apache.org>
Received-SPF: neutral (nike.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: 
 <120223801.6201751298294211434.JavaMail.root@spooler6-g27.priv.proxad.net>
References: <AANLkTi=mdg+ZWuFsPifyVMdADf_izQRGpSwvj9ir9LhZ@mail.gmail.com>
 <120223801.6201751298294211434.JavaMail.root@spooler6-g27.priv.proxad.net>
From: Mikkel Meyer Andersen <mikl@mikl.dk>
Date: Mon, 21 Feb 2011 15:48:19 +0100
Message-ID: <AANLkTi=i4pCVwZk8pgbCerd57KSCABs487PaJpQbyVYP@mail.gmail.com>
Subject: Re: [MATH] Matrix hierarchy reorganization
To: Commons Developers List <dev@commons.apache.org>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

2011/2/21  <luc.maisonobe@free.fr>:
> Hi Mikkel,
>
> ----- "Mikkel Meyer Andersen" <mikl@mikl.dk> a =E9crit :
>
>> Dear community,
>>
>> I've started on implementing Kalman filter and have once again run
>> into a structural challenge in regards to matrix hierarchy (as with
>> https://issues.apache.org/jira/browse/MATH-435 and
>> https://issues.apache.org/jira/browse/MATH-437 ).
>>
>> We have two base types of matrices: RealMatrix and FieldMatrix, both
>> extending AnyMatrix. Both RealMatrix and FieldMatrix have a lot of
>> the
>> same functionality, e.g. add, subtract, scalarAdd, scalarMultiply,
>> multiply, preMultiply etc.
>>
>> I think it would be beneficial to try to move these standard matrix
>> methods to AnyMatrix, or at least introduce a new intermediate
>> AnyAlgebraicMatrix (or similar). It would be natural in several
>> aspects to just require an AnyAlgebraicMatrix, e.g. with Kalman
>> filter
>> or repeated squaring, instead of forcing the user to use either
>> RealMatrix or FieldMatrix (or implement almost similar code for
>> both).
>>
>> What do you think about such a reorganization of the matrix
>> hierarchy?
>
> I think we really need to reorganize our linear algebra, but am not sure =
about
> this specific change, as I fear it would deter performances (which alread=
y
> needs improvements).
>
> At ApacheCon Atlanta, we had a short discussion with Isabel Drost form th=
e Mahout
> project. We would really much have liked to have Ted Tunning around too
> but he was not there. We said to Isabel that once we have 2.2 out and
> will be reflecting about 3.0, we would like to have some requirements fro=
m
> the Mahout project. We were not able up to now to satisfy their needs bec=
ause
> of backward compatibility. However, now is the time we can change anythin=
g
> we want.
>
> The current problems I see with our linear algebra package are the follow=
ing ones:
> =A0- clumsy hierarchy
> =A0- inefficient for large matrices (say above 4000 rows and columns)
> =A0- insufficient support for sparse matrices (most algorithms simply don=
't use sparsity)
> =A0- SVD is still not stable, after 3 complete rewrites
> =A0- missing features (eigen decomposition for non-symetric matrices)
>
> The cool features we have are:
> =A0- support for field matrices (this is really useful with Complex eleme=
nts,
> =A0 with Fraction elements when we need exact computation of polynomial c=
oefficients for
> =A0 example, and with Dfp elements for high accuracy)
> =A0- reasonable API for decomposition, mainly Jama compatible but slightl=
y better IMHO
>
> The things we can change at 3.0 are:
> =A0- API
> =A0- implementation
>
> The things we will for sure have some difficult discussions are:
> =A0- dependencies
> =A0 (I guess Phil and myself don't like dependencies, whereas Ted and Gil=
les
> =A0 =A0are happy with them)
>
> What I would like to see:
> =A0- more exchanges with the Mahout community
>
> I'm not sure if we should have this discussion here, or on the Mahout lis=
t or
> on both. Ted, what to you think about this, are their a sufficient number=
 of
> Mahout people here or should we cross-post everything ?
>
> Thanks for bringing up this really important topic Mikkel!
> Luc
>
>>
>> Cheers, Mikkel.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>

Hi Luc,

It wasn't my meaning to open such huge a discussion at the moment :-).
Sorry for that. I think we should postpone this discussion until 2.2
is out. Meanwhile, I'll be thinking about possible ways to go with the
hierarchy.

Cheers, Mikkel.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org