arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Antoine Pitrou <anto...@python.org>
Subject Re: [DISCUSS][C++][Proposal] Threading engine for Arrow
Date Fri, 03 May 2019 08:53:43 GMT

Hi Jed,

Le 03/05/2019 à 05:47, Jed Brown a écrit :
> I would caution to please not commit to the MKL/BLAS model in which the
> library creates threads internally.  It's a disaster for managing
> oversubscription and affinity issues among groups of threads and/or
> multiple processes (e.g., MPI).

Implicit multi-threading is important for user-friendliness reasons
(especially in higher-level bindings such as the Python-bindings).

> The library is then free to use constructs like omp taskgroup/taskloop
> as granularity warrants; it will never utilize threads that the
> application didn't explicitly give it.

I don't think we're planning to use OpenMP in Arrow, though Wes probably
has a better answer.

Regards

Antoine.

Mime
View raw message