mxnet-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [incubator-mxnet] cloudhan commented on issue #9686: [Discussion] MXNet 2.0 Roadmap (was: APIs that might be a good idea to break in 2.0)
Date Wed, 17 Jul 2019 12:26:03 GMT
cloudhan commented on issue #9686: [Discussion] MXNet 2.0 Roadmap (was: APIs that might be
a good idea to break in 2.0)
URL: https://github.com/apache/incubator-mxnet/issues/9686#issuecomment-512232079
 
 
   > I think we should provide a user-friendly thread-safe inference API for deploying
in c++, java, etc. We can focus on naive engine in inference since it's very hard to refactor
threaded engine to be thread-safe. A good and easy-to-use executor should have the following
properties:
   > 
   > * One instance of the executor is enough for multi-threaded inference, which means
it can be used simultaneously in different threads.
   > * The immutable ndarray in executor should be shared in multi-threaded inference to
save memory footprint.
   > 
   > Now we have `MXPredCreateMultiThread` in C API, but it's buggy and we still need to
create multiple executors for each thread.
   
   Sounds like refactor execution with TBB and adding some buffering mechanism´╝č

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message