commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Phil Steitz <>
Subject Re: [math] getting changes included into commons-math (was Re: Home for the colt fork)
Date Thu, 10 Dec 2009 11:37:43 GMT
Ted Dunning wrote:
> Similarly, had code from a
> contributor and got totally shot down basically with "We don't care what you
> think, we don't want it".  Extensive attempts on my part to find common
> ground just got shot down by Phil with -1 and essentially no more discussion
> than "don't like it".

Sorry you feel that way.  My objections were to the design, not in
any way personal or dismissive.  I did indicate alternative ways to
get the desired functionality implemented.  I also explained, I
think, pretty clearly why "I did not like it" and I stand behind my

> Mahout has some unusual requirements and definitely can't use Math as is.
> It is also fast moving and needs changes to actually happen.  The experience
> has been that only totally trivial non-changes get accepted (
> and
> are examples).  Anything more
> interesting seems to get bogged down in NIH or endless alternations of
> "not-now we are about to have a major release and can't be distracted" and
> "not-now because we aren't doing a major release and have stay compatible".
> On Wed, Dec 9, 2009 at 11:58 AM, Jake Mannix <> wrote:
>> On Wed, Dec 9, 2009 at 11:09 AM, Benson Margulies <
>>> wrote:
>>> This is interesting. We have a raft of mathematically qualified
>>> committers on Mahout, and this message asking for help on
>>> commons-math, and a raft of code marooned at mahout that wants to be
>>> in commons math. If I were one of those mathematically competant
>>> individuals, I'd be off attaching a patch or three to a JIRA or two
>> The commons-math linear APIs have been described as effectively locked
>> until 3.0, due to back-compat requirements.  This means that any code
>> contributed
>> into c-math would live in a parallel (no pun intended) to the linear
>> primitives which
>> exist already in there.
>> Adopting something like MTJ or Colt in Mahout turned out to be easier,
>> because
>> we are on release 0.2 (heading for 0.3 now), and have less stringent
>> back-compat
>> requirements, so we are overhauling our linear apis (read: even user-facing
>> interface changes) to take advantage of useful parts of Colt, and are
>> planning on
>> using our Colt fork as the underlying implementation.
>> Commons-math expressed that changing linear APIs is not something they can
>> do,
>> given the maturity of their library, so where would Colt *go* in c-math?
>> It's own
>> submodule, having its own eigendecompositions and svd and so forth, running
>> parallel to the current c-math impls?  Why?
>> Who would maintain it and write tests for it, and how do you explain to
>> end-users which they should use?
>>> On Wed, Dec 9, 2009 at 1:48 PM, Luc Maisonobe <>
>>> wrote:
>>>> Ted Dunning a écrit :
>>>>> Actually, the reason that we have Colt in Mahout is it has proven
>>> impossible
>>>>> to get changes into commons math.  We really, really wanted to use
>>> commons
>>>>> math rather than have our own linear algebra package, but it just
>> proved
>>>>> impossible and we didn't want to wait forever.
>>>> If you really, really wants to use commons math and want changes to be
>>>> included, contribute them.
>> I have submitted patches for the following tickets: MATH-312 (and
>> acceptance
>> of that patch blocks my patch for MATH-314), MATH-316 and MATH-317, none
>> of which have appear to have had much progress on.  All of my patches come
>> with unit tests for new functionality.
>> On the other hand, when I opened the discussion about extending the
>> functions
>> package to enable composable functions (MATH-313), I got an entirely
>> hostile
>> response, which only tempered as far as "+0" on adding it after discussion.
>> In particular, my first step at making commons-math something Mahout could
>> standardize on for linear work was MATH-312, which I did submit a patch
>> for,
>> and revised it many times after discussion about what is acceptable
>> practice
>> in c-math.  Not yet applied, months later.  It's probably far out of date
>> now...
>> Similarly, when I tried to ask what the status on decisions on whether to
>> adopt
>> MTJ or Colt, the statement by Phil was basically that commons-math would
>> not
>> adopt anything which had any external dependencies or
>> not-easily-human-readable java source (which ruled out MTJ because of f2j
>> produced code), and which had to be fully tested and maintained prior to
>> adoption (which rules out Colt which has no unit tests yet).
>> Ted and I weren't making "requests" for other people to do work, we were
>> wondering whether even offers to do some of the work would be accepted,
>> and for many of the questions/suggestions we had, it seems the desires
>> and requirements of the Mahout community were incompatible with those
>> of commons-math.
>>  -jake
>>>  > I think the only change that was proposed and not done because of lack
>>>> of consensus was the inclusion of MTJ (and I don't consider the
>>>> discussion closed on that topic either, so it may still happen some
>>>> day). All the other changes that are desired are simply lacking someone
>>>> to do the work. There were proposals to extend the linear algebra API,
>>>> proposals to add more support for sparse matrices, proposals to get
>>>> partial decomposition ... But sparse contributions (pun intended).
>>>> I try to do what I can, but as you have probably seen have been rather
>>>> silent since 2.0 release. For my part, I really, really need help. I
>>>> would like to fix the problems in the eigen decomposition and SVD but
>>>> need a good kick to get on it, and having only requests and no help is
>>>> not really motivating.
>>>> Luc
>>>>> If that problem were solved, then it would be great to depend on
>> commons
>>>>> math.  If that problem isn't solved, then there is no way to depend on
>>>>> commons math.
>>>>> On Tue, Dec 8, 2009 at 6:19 AM, Benson Margulies <
>>>> wrote:
>>>>>> We can't possibly have a dependency on Mahout in the long term.
>> Either
>>>>>> we all go shares on code in some other piece of commons, or we end
>>>>>> with two forks, which would be sad.
>>>>>> On Tue, Dec 8, 2009 at 8:33 AM, James Carman <
>>>>>> wrote:
>>>>>>> I wouldn't like to see a dependency on mahout code in a "commons"
>>>>>>> library.  That seems kind of backwards.  If Mahout wants to offload
>>>>>>> this stuff, we can move it into a library in commons (which is
>>>>>>> typically how stuff used to happen in Jakarta).
>>>>>>> On Tue, Dec 8, 2009 at 8:18 AM, Benson Margulies <
>>>>>> wrote:
>>>>>>>> Mahout now has a fork of a portion of the 'category A' portion
>> the
>>>>>>>> CERN colt library forked. The Mahout fork is, of course,
in the
>>> Mahout
>>>>>>>> tree under a Mahout Java package and Maven triple.
>>>>>>>> I want to use the collections classes from Mahout as the
core to a
>>> new
>>>>>>>> set of commons-primitives classes that do the useful things
>> GNU
>>>>>>>> Trove does.
>>>>>>>> The classes I want to start from depend on the classes that
are in
>>> the
>>>>>>>> Mahout fork.
>>>>>>>> As a temporary expedient, I can depend on their there. However,
>>>>>>>> submit that it would be more better if the mathematical code
>> in
>>>>>>>> commons-math. Was this option explored?
>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail:
>>>>>>>> For additional commands, e-mail:
>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail:
>>>>>>> For additional commands, e-mail:
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail:
>>>>>> For additional commands, e-mail:
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail:
>>>> For additional commands, e-mail:
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail:
>>> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message