commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gilles <>
Subject Re: [Math] Commons Math (r)evolution
Date Fri, 10 Jun 2016 22:39:30 GMT
Hello Jörg.

On Fri, 10 Jun 2016 12:19:56 +0200, Jörg Schaible wrote:
> Hi Gilles,
> Gilles wrote:
>> Hi.
>> On Fri, 10 Jun 2016 01:48:20 +0200, Jörg Schaible wrote:
> [snip]
>>> MATH-172 is about an enhancement. Unfortunately no-one can 
>>> currently
>>> implement it, so we have to wait until someone can or the issue 
>>> stays
>>> simply
>>> unresolved again. You've requested for help and that was the proper
>>> action.
>>> However, there has been no problem to release 3.0 in this state, so
>>> why
>>> should it be a problem now for 4.0?
>> Because the context has changed: in 3.0 there was support, now there
>> isn't.
> That does not state the fact, that the code is already released and 
> it does
> not matter at all if users ask questions for release 3.0 or 4.0.

In practice, you may be right; but I think that it does matter in a
situation such as this one (which is not common - at least, it's the
first time I see a fork).

Let me recall that the [math] consensus has been (for years): one line
of development.
This was changed, without formal consensus, supposedly in order to
_support_ legacy users.

In my mind this support was acceptable for bug-fixes only (according
to the consensus).

This support can still exist and, for v3.6.x, I agree with you.

But for v4.0, I'm not going to give one more minute to maintain code
that is either bad, or outdated.

As others have said, we lost contributors on that fanaticism about
unproven scenarios.

If you want to keep CM as is (for who knows what future), that's fine
for you and anyone who is happy working with that code.

I'm not happy with the status quo.
The fork could have been an opportunity to try something new.

>>> MATH-1735 is a bug report for a problem which is currently not
>>> reproducible.
>>> Again you did the right action, but without more input and without 
>>> a
>>> possibility to reproduce the problem, there's not much we can do.
>>> Again, why
>>> should this issue prevent a release of 4.0?
>> The code in question should not have been in CM. [That was my 
>> position
>> back
>> then (cf. archive).]
> Again, you cannot change history, it is already released. And a new 
> release
> of the code does not make the situation worse.

Every major release of Commons Math broke (sometimes many) things.
If it didn't, a major release wouldn't have been necessary.

Not making a new release of <some code> is in no way taking something
from the users.

>> And every bug report for it is a reminder that unmaintainable code
>> should
>> not be released.
> That is your interpretation. For me it is a normal bug report and we 
> can
> eigher solve it on our own or have to wait for a contribution.

So, IIUC, you'd release knowingly buggy code.
That makes no sense to me; so, I'm not going to be part of this.

>>>> Moreover what could be true for VFS is not for CM where there are
>>>> many,
>>>> many different areas that have nothing in common (except perhaps
>>>> some
>>>> ubiquitous very-low utilities which might be worth their own
>>>> component
>>>> to serve as a, maybe "internal", dependency).
>>>> Also, compare the source basic statistics (lines of code):
>>>>                VFS      CM
>>>> Java code    24215   90834
>>>> Unit tests    8926   95595
>>>> All in all, CM is more than 5 times larger than VFS (not even
>>>> counting
>>>> documentation).
>>> Any why is size suddenly a problem?
>> Not suddenly.
>> I've raised this issue for years.  Please look at the archives!
> Then: Why is size a problem? It is only a problem if *you* try to 
> support
> all of it at once. Nobody expects that.

I, as a user, expect that.

I chose to use Commons Math, and advertised it to others (to my
current dismay), because of the *outstanding* support I knew I
would get in case of bugs or requests for enhancement.

I do not want to work on a project that is either going to die,
or become legacy (and then dead/dormant/whatever).

> [snip]
>>> Fine. But talk about artifacts, not components. Apache Commons Math
>>> is still
>>> *one* component of Apache Commons. It does not matter if you divide
>>> the code
>>> into different artifacts as long as anything is released together.
>> I know.
>> What you can't seem to get to is that I consider it a bad idea to
>> release
>> unsupported code.
> You've stated that multiple times now.
>> I think that "I do not know (and nobody else does)" is not an
>> acceptable
>> answer to user requests.
> See, this is the difference. To me this *is* acceptible. Especially 
> if users
> have to be kind of experts themselves to use this code.

I don't agree with that.

Moreover, how could a maintainer be confident that a fix provided by 
user is correct if he doesn't have the slightest idea about the code?

To get to any level of confidence starting from scratch is a huge time
sink; I don't have this time anymore.

>> If we _know_ that some parts of the code would be unsupported (such
>> that it
>> would elicit this kind of answer for bug reports), then it's
>> _deceiving_ to
>> release that code.
> A new release does not change the situation at all. With such a 
> definition
> we could move quite some of our components directly to the attic, 
> because
> the original authors are no longer around and we might currently have 
> no
> expert around.

Maybe, maybe not.
One size does not fit all.

Let's see if you can pick any other component that would have the same
level of discrepancy between the codebase, the likelihood that bugs 
be raised, and that there is no one around to fix them (or even to 
the reporter on how to go about a solution or workaround), as there is
for CM.

>>> Individual release cycles for the different parts can only happen 
>>> if
>>> Math is
>>> TLP, but not in Apache Commons. We will certainly not allow and
>>> create again
>>> any sub-umbrellas (search the Jakarta archives).
>> Who talked about sub-umbrellas?  I didn't.
> I talk about it, because it is what the result looks to me.

It's not what I advocate.

Right now I expect to easily create one component with the contents
of the "o.a.c.m.rng" package.

As I've explained: It is general-purpose and I'd see as absurd that
Commons people argue that a true component should not be a Commons
component just because I developed it inside the [math] repository.

The next cases in line (in line because there are people (2) willing
to create the code and support it) are:
  * Complex numbers
  * Clustering algorithms

They might be less easy to port, but at the "user" level they are
totally independent (if we leave out some of the CM requirements
that are subject to revision by the project developers).

>> I've explained that
>>   1. CM cannot be released as it was before
> You've expressed that *you* don't want to release it as it was before 
> (well-
> founded however).
>>   2. for some parts, the necessary _minimal_ support has 
>> "disappeared"
> That does not immediately invalidate the existing code.

Never said it did.

You seem to look to the past (released), where I want to look to the
future (new development and modernization of the library).

> One fact that you
> don't want to see.

Not true.  Cf. above (3.6.x vs 4.0).

>>   3. some parts can be turned into independent components (with 
>> _full_
>>      support)
> Have we some kind of agreement here to introduce new Commons 
> components?

That's the main question.

Do I have to initiate a VOTE for it?

Do I need other clearances in order to request "git" repositories from

> As
> far as I know, we just did not oppose to break Math into individual
> artifacts.

I won't do that; given the circumstances, it is nonsense to waste
one's time just to have unsupported "modules".
The monolithic nature of CM is but one problem.
Some parts of it are simply good to throw away.

Moreover there are plenty of advantages to real components over modules
whenever feasible.

>>   4. Some people are ready to perform the necessary steps in order 
>> to
>>      create these components
> The work on the code is still independent of a monolithic Math 
> component vs.
> inidividual (let's say) Maven projects. The refactoring is the same 
> if you
> simply want to minimize the dependencies between the Java packages in 
> Math.

No, it's not.
Making CM multi-modules is much more ambitious than making one or two
Moreover it is focusing on form rather than contents.

>> Please comment on how this contradicts Commons policy.
> The ASF in general does not like umbrella TLPs at all. Apache Commons
> actually has been near this category all the times. We try to manage 
> a
> (large) set of general purpose libraries with the intent to provide 
> long
> time compatibility (sometimes possibly exaggerated) or provide clear 
> upgrade
> paths if a redesign was necessary. We do this, because we are aware 
> that our
> components are nearly always part of large software stacks and we may 
> impact
> a lot of stuff if we fail.

Does not apply to CM.  Cf. previous post about supposedly mature code.

> However every of our components has a general purpose. Math is 
> described as
> "Lightweight, self-contained mathematics and statistics components". 
> The
> component in its completeness is certainly no longer lightwight.

Surely, it's not lightweight.
It also has no single purpose.

It was a mistake to keep it as a single Commons component in v2.0 and

Is it too late to fix that bug?

> What you propose now, is to move Math in its current state into 
> dormant (or
> even attic, because it will then never be revived in its current 
> state) and
> create several (how many?) new Commons components,

I can speak only for what I think I can achieve: one component with
the (uniformly distributed) random generators.

A few other components are probably worth trying but it depends on
the goal(s).

> all of them focussed to a
> special mathematical problem. And this is exactly the reason why a 
> bunch of
> new Math components no longer fit into Commons, because they fail the
> "general purpose".

That's not true.
Each component would be general-purpose (or none of the Commons 
can fit that definition either), not focused on a specific problem 
or not).

  * Random number generators are used in... games
  * Complex number are used... in computer-generated artwork (fractals)
  * Clustering can be used to classify... anything.

> It has happened before that a Commons component had outgrown the 
> general
> purpose (see former HttpClient that is now a successful TLP) and Math 
> might
> be not the last one. Remember, we already had a successful vote for a 
> Math
> TLP this year for exactly these reasons.

Some codes could truly become great components even if a TLP existed.

If you don't want them , that's something I'll have to take into


> [snip]
>>> What you talk about is a complete new component without a proper
>>> upgrade
>>> path for users of Math 3.x.
>> "Commons Math" is a dead end.  Is it that which you did not get?
> It's death when we decide it together. Currently it's your 
> (well-founded)
> opinion.
>> There is "Commons" and there are "math codes" implementing various
>> functionalities, some in good shape, which I want to help release,
>> some in bad shape (for numerous reasons which you don't know about
>> and don't seem to want to hear about) which I'm not going to help
>> release as long as they are in that state.
>>> You wanna use the new stuff for complex numbers?
>>> Well, adjust your code to the new structures in
>>> commons-math-complex-4.0.
>> No.  It should be "commons-complex-numbers-1.0" (or perhaps "0.1" if
>> that's allowed).
> See, I doubt that all devs were aware of the fact, that we're 
> currently not
> just talking about a Math component with multiple artifacts, but a 
> bunch of
> new Commons components instead (discontinuing the "Math component").
>>> Oh, you also want to use genetic algorithms?, Well, that part of 
>>> the
>>> code
>>> should stick with commons-math-3.0, because we did not took it over
>>> for 4.0.
>>> Maybe for 4.1 ...
>>> Sorry, but for such a release I already propose my vote today: -1
>> So you prefer to sink whole work rather than be pragmatic about the
>> new reality.
> As PMC I care for a sane Commons community with already ~40 
> maintained
> components.
>> How is that preferable to the user community?
> And what does the name "commons-complex-numbers-0.1" vs. 
> "commons-math-
> complex-number-4.0" change for the users' situation I've described 
> above?
>>> In that case I'd move Math really into the incubator.
>> Please do.
>> There are several options:
>>   1. Go on with a monolithic Commons Math
>>   2. Break the codebase into (maven) modules
> Concerning the release cycle, there's no difference between 1 and 2.
>>   3. Create (in the long term) an all-encompassing TLP
>>   4. Create reusable components (right now)
> IMHO, 4 is not an option for Commons, only for a Math TLP. Which 
> needs an
> own community. Which has to be rebuilt.
>>   [Add others if appropriate.]
>> Which is more useful, in your opinion?
> As stated above.
>> To which are you going to contribute (multiple choice allowed,
>> of course)?
> Nowadays I typically care for the release only. And I care for the 
> impact of
> it, because my software stack is also built upon Commons.
> Cheers,
> Jörg

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message