mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pedro Larroy <pedro.larroy.li...@gmail.com>
Subject Re: MXNet and Cython
Date Thu, 15 Feb 2018 11:52:22 GMT
I don't know about the speed, and it shouldn't be the only criteria to
do things. My main question is: Why Cython instead of C++, is there
data or a prototype that would show light into what is the best
decision here?

Regarding the bindings, as with boost::python seems it's a very thin
wrapper over CPython. Irregardless of the bindings used to interface
between C++ and Python. I think my main point was: why not push the
slow parts of ndarray indexing to C++ instead of Cython, what are the
tradeoffs? can this code be maintained well and debugged? How is this
Cython code going to impact packaging or native MNet on devices and
other language bindings which won't be able to use this logic and
improvements. If it were done in C++ it would be reusable across
language bindings and from native applications. There are users
wanting to use MXNet from other platforms, from native to Java, one of
the strong points of MXNet is that is not just Python centric and it's
portable.

As you might now we are not yet testing right now in devices on CI due
to the huge work that was improving the main CI, adding C++ tests etc.
So right now code and functionalities are getting pushed in that break
builds and compatibility across platforms. I think we should look
further into the tradeoffs of introducing breaking functionalities or
new technologies such as Cython that could potentially be problematic
and validate with a prototype before just being pushed into master and
then letting the community having to deal with this post-facto. And
addressing technical as well, specially before introducing more
complexity.

Pedro.



On Wed, Feb 14, 2018 at 9:17 PM, Chris Olivier <cjolivier01@gmail.com> wrote:
> *I don't see in pybind11 that it's claiming to be especially performant,
> only that it's convenient in passing data back and forth.*
>
> *Is it claimed somewhere that it is especially fast?*
>
> On Wed, Feb 14, 2018 at 10:45 AM, Pedro Larroy <pedro.larroy.lists@gmail.com
>> wrote:
>
>> Why Cython and not just C++ with simple python bindings like
>> https://github.com/pybind/pybind11 ? What's the motivation?
>>
>> Have we considered the trade offs between maintainability, speed and
>> tooling? Also this Cython code is not going to be portable as it would
>> be in C++, and has some learning curve and tooling / portability
>> overhead.
>>
>> It could also complicate using MXNet from Brazil.
>>
>> Are there details somewhere of what is going to be converted to
>> Cython? Are there any measurements or profiling that indicate that
>> rewriting some parts in Cython is the best solution to some bottleneck
>> or slow code?
>>
>> Pedro.
>>
>> On Mon, Feb 12, 2018 at 8:57 PM, Chris Olivier <cjolivier01@gmail.com>
>> wrote:
>> > Hi all,
>> >
>> > I am starting to look into converting some of the python code to Cython.
>> >
>> > Any comments, suggestions, past experiences, are welcome.  Can reply to
>> > this thread, or I have an editable document here:
>> >
>> > https://docs.google.com/document/d/1pvW5nYbf3pbiak95xZdqgLbrjQdvV
>> rVGoYdiU3hNsqc/edit?usp=sharing
>> >
>> > So far, the JIRA items are part of this epic:
>> >   https://issues.apache.org/jira/browse/MXNET-6
>> >
>> > -Chris
>>

Mime
View raw message