commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikkel Meyer Andersen <m...@mikl.dk>
Subject Re: [MATH] Matrix hierarchy reorganization
Date Mon, 21 Feb 2011 14:48:19 GMT
2011/2/21  <luc.maisonobe@free.fr>:
> Hi Mikkel,
>
> ----- "Mikkel Meyer Andersen" <mikl@mikl.dk> a écrit :
>
>> Dear community,
>>
>> I've started on implementing Kalman filter and have once again run
>> into a structural challenge in regards to matrix hierarchy (as with
>> https://issues.apache.org/jira/browse/MATH-435 and
>> https://issues.apache.org/jira/browse/MATH-437 ).
>>
>> We have two base types of matrices: RealMatrix and FieldMatrix, both
>> extending AnyMatrix. Both RealMatrix and FieldMatrix have a lot of
>> the
>> same functionality, e.g. add, subtract, scalarAdd, scalarMultiply,
>> multiply, preMultiply etc.
>>
>> I think it would be beneficial to try to move these standard matrix
>> methods to AnyMatrix, or at least introduce a new intermediate
>> AnyAlgebraicMatrix (or similar). It would be natural in several
>> aspects to just require an AnyAlgebraicMatrix, e.g. with Kalman
>> filter
>> or repeated squaring, instead of forcing the user to use either
>> RealMatrix or FieldMatrix (or implement almost similar code for
>> both).
>>
>> What do you think about such a reorganization of the matrix
>> hierarchy?
>
> I think we really need to reorganize our linear algebra, but am not sure about
> this specific change, as I fear it would deter performances (which already
> needs improvements).
>
> At ApacheCon Atlanta, we had a short discussion with Isabel Drost form the Mahout
> project. We would really much have liked to have Ted Tunning around too
> but he was not there. We said to Isabel that once we have 2.2 out and
> will be reflecting about 3.0, we would like to have some requirements from
> the Mahout project. We were not able up to now to satisfy their needs because
> of backward compatibility. However, now is the time we can change anything
> we want.
>
> The current problems I see with our linear algebra package are the following ones:
>  - clumsy hierarchy
>  - inefficient for large matrices (say above 4000 rows and columns)
>  - insufficient support for sparse matrices (most algorithms simply don't use sparsity)
>  - SVD is still not stable, after 3 complete rewrites
>  - missing features (eigen decomposition for non-symetric matrices)
>
> The cool features we have are:
>  - support for field matrices (this is really useful with Complex elements,
>   with Fraction elements when we need exact computation of polynomial coefficients for
>   example, and with Dfp elements for high accuracy)
>  - reasonable API for decomposition, mainly Jama compatible but slightly better IMHO
>
> The things we can change at 3.0 are:
>  - API
>  - implementation
>
> The things we will for sure have some difficult discussions are:
>  - dependencies
>   (I guess Phil and myself don't like dependencies, whereas Ted and Gilles
>    are happy with them)
>
> What I would like to see:
>  - more exchanges with the Mahout community
>
> I'm not sure if we should have this discussion here, or on the Mahout list or
> on both. Ted, what to you think about this, are their a sufficient number of
> Mahout people here or should we cross-post everything ?
>
> Thanks for bringing up this really important topic Mikkel!
> Luc
>
>>
>> Cheers, Mikkel.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>

Hi Luc,

It wasn't my meaning to open such huge a discussion at the moment :-).
Sorry for that. I think we should postpone this discussion until 2.2
is out. Meanwhile, I'll be thinking about possible ways to go with the
hierarchy.

Cheers, Mikkel.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Mime
View raw message