Mailing-List: contact dev-help@commons.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Commons Developers List" <dev@commons.apache.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: 7bit
Date: Wed, 23 Aug 2017 03:14:59 +0200
From: Gilles <gilles@harfang.homelinux.org>
To: <dev@commons.apache.org>
Subject: Re: [All][Math] New component: "Commons =?UTF-8?Q?Geometry=22=3F?=
In-Reply-To: <CAF8HOZLHmR6XdK=WX+FehtzcCgFTkjUQeSLwOq5cLtEQuDU8Zw@mail.gmail.com>
References: <715027bee0b4681d76bd15328fad2415@scarlet.be>
 <CAF8HOZJ3VhsgrYtSNx3U1OJFNfcmPCyV==h9L7qxtc+7PpYJ0Q@mail.gmail.com>
 <2B12843C-A5E3-4E35-A742-2C11C248AD89@dslextreme.com>
 <506364C0-756B-4728-9D3F-8943BDB141AF@apache.org>
 <3d838fcb88a8a229893ee31e944e4992@scarlet.be>
 <9D35F1FE-5AEA-4467-AD0E-B18102D431F5@dslextreme.com>
 <2442f97c7dab42de8c3205313c05426b@scarlet.be>
 <CAF8HOZLHmR6XdK=WX+FehtzcCgFTkjUQeSLwOq5cLtEQuDU8Zw@mail.gmail.com>
Message-ID: <38b52a5993e88751b049e065900f9ac7@scarlet.be>
User-Agent: Scarlet Webmail
X-DCC-scarlet.be-Metrics: sif; whitelist
X-Virus-Scanned: clamav-milter 0.98.1-exp at sif
X-Virus-Status: Clean
archived-at: Wed, 23 Aug 2017 01:15:12 -0000

On Tue, 22 Aug 2017 18:35:22 +0200, Jochen Wiedmann wrote:
> On Mon, Aug 21, 2017 at 11:20 PM, Gilles
> <gilles@harfang.homelinux.org> wrote:
>
>> Other programming languages eco-systems successfully follow
>> an approach based on (really) small components; why would you
>> want "Commons Math" to remain this monolithic beast?
>
> No one is arguing for monolithic. We are simply questioning, whether 
> a
> split in Maven modules is sufficient, or not.

It is not.

>
> My impression is, that you are not even considering that more
> lightweight approach.

I have. From the outset.
I'm going to reiterate, once more.[1]

Some of the packages/codes of CM are candidates for standalone
components.
There are several reasons, possibly different for different
candidates:
1. Low-level, e.g.
    * Check for equality between "double" values
    * Fractions
    * Complex numbers
    * Combinatorics
    * Prime numbers
    Each of those can be of interest to quite different categories
    of users.
    [However, being quite low-level, it was not a bad compromise to
    group them all in a component called "Numbers" on the rationale
    that all deal with some kind of "number" (we can look at it as
    the level just above the elementary "Number"s of the JDK).
2. Self-contained (i.e. no dependency on anything else), e.g.
    * Code originally in the "o.a.c.math4.random" package that has
      started the development of "Commons RNG".
      Again of interest to a wide range of users.[2]
3. Standalone (with dependencies on a few low-level utilities
    such as those that are, or would fit, in "Numbers", and "RNG",
    but on nothing else from CM), e.g.
    * Package "o.a.c.math4.geometry"
    * Package "o.a.c.math4.ml"
    * Package "o.a.c.math4.distribution" (except perhaps the
      "multivariate" Gaussian)
    Those are, by all sensible criteria, independent fields of
    expertise and different user/developer communities; there is
    no reason to group them in a single library.

With those, ends the list of what I think would be good
components (with no less general usefulness[3] than some
other "Commons" components).
No "avalanche" of new components, no unbearable "noise"
on the ML, no fear of PMC exhaustion at validating
release candidates.
I contend that project management and review work will
be _easier_ due to the more focused subject matters.
And I've no problem with doing one after the other, as
said previously.

Then there is package "o.a.c.m.genetics", whose support
should be discontinued (IMHO), for reasons given elsewhere.

The rest of CM is comprised of package with intertwined
dependencies:
  * matrix algebra (package "o.a.c.m.linear")
  * functions, integration, interpolation, root solvers
    (package "o.a.c.m.analysis")
  * differential equation solvers (package "o.a.c.m.ode")
  * statistics (package "o.a.c.m.stat")
  * optimizers (packages "o.a.c.m.optim" and "o.a.c.m.fitting")
  * automatic differentiation ("o.a.c.m.analysis.differentiation")

In addition to being the sort of functionality that indeed
constitutes a "math toolbox", those packages would also stay
together for the following practical reasons:
1. Most unresolved issues target codes in them. Some have
    been open for years.  Some seem to require non-trivial
    debugging.
2. Some of those issues can't be solved without significant
    refactoring (particularly in "linear" and "stat").
3. Some codes were left dangling midway of a refactoring
    (namely "optim").

One of my points is to make a clear and useful difference
between actively supported code (the new components) and
code in need of new blood ("MathLegacy").
The former is timely released with all bugs fixed.
The latter could be released (PMC willing) as a WIP, to
not let down users of codes that satisfy their needs (i.e.
the bugs do not show up for them).

Down to the level of practicalities, it will of course be
an improvement of "Mathlegacy" if it can be modularized.
But this in itself is already a lot of work which it would
be insane to complete without also fixing the design bugs
as they are encountered.

> The arguments are giving, are typically answered
> just as well with modules.

If that were true, the same argument would apply to the
whole of "Commons": just release all of it as modules
within a single maven project.

> Plus, you avoid losing oversight over the
> sum.

Adding apples and pears.
Oversight of unrelated functionalities is useless (and even
a liability, as it turned out).

Oversight is required at the "Commons" level, to ensure that
components are healthy.

What other proof do you need that "Commons Math" wasn't?[4]

Plus, a team of maintainers who, together, had a nearly
100% reactivity level could make up for the project's
failure to define a clear "management"; now the reactivity
level is on life-support,[5] and the shortcomings should
be apparent to all.


Gilles

>
> Jochen

[1] Those reasonable arguments are already in the archive in
     one form or another...
[2] If they have strong randomization requirements, they should
     stop using "java.util.Random". See
       
http://commons.apache.org/proper/commons-rng/userguide/rng.html#a5._Quality
[3] We should at least agree to disagree on what is "generally"
     useful, when there is nothing more than vacuous arguments
     like "It's math-related".  Perhaps another way to express
     what I mean: Google turns up
       * About 27.400.000 results for "Wikipedia math"
       * About 1.970.000 results for "Wikipedia java language"
[4] I'm still waiting for a reaction to that post:
       http://markmail.org/message/uiljlf63uucnfyy2
[5] For months (December 2015 up to the announcement of the fork)
     I've been the only one to answer user requests. After the
     fork, I tried to draw your attention on the fact that the
     situation was not sustainable, and offered a viable plan.
     No one proposed an alternative (which they would be ready
     to actually pursue).


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org