commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Phil Steitz <>
Subject Re: [Math] Little thought about multi-threading
Date Sun, 22 Jul 2012 16:27:17 GMT
On 7/21/12 6:17 AM, Gilles Sadowski wrote:
> Hi.
> My previous post (with subject "Synchronisation") made me think (again) that
> it might be useful to start considering how to take advantage of
> multi-threading in Commons Math.
> Indeed, it seems that some parts of the library might end up not being used
> anymore because their performance simply cannot match competing
> implementations that do benefit form parallelization. [The recent example
> that comes to mind is the FFT.]

This is an interesting question.  I am also -1 on adding
dependencies, but it would be a good idea to look at how others have
solved the problem of how to support parallel execution by multiple
threads without managing threads directly.  Lots of [math]
algorithms could be parallelized.  The question is how to
effectively coordinate the work without owning or creating the
workers.  I would be -0 to any suggestion that involved [math]
itself spawning threads, since that 0) creates management headeaches
1) may violate some container contracts and 2) forces execution
threads to be in the same process.  I think it is worth thinking
about how we might support parallel execution by externally managed
workers.  An obvious thing to look at is how to break our
parallelizable algorithms into pieces that could be executed in
Hadoop Map/Reduce jobs.  Step 0) is the breaking up part.  Then step
1) might be either some examples added to the user guide or custom
Pig functions (or examples of how to code them).

> Best regards,
> Gilles
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message