tvm-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [tvm] AndrewZhaoLuo edited a comment on pull request #8363: [Tuning] Allow multiprocessing spawn to work (on macOS llvm at least)
Date Wed, 30 Jun 2021 17:47:33 GMT

AndrewZhaoLuo edited a comment on pull request #8363:
URL: https://github.com/apache/tvm/pull/8363#issuecomment-871606570


   The main lesson here is if we use python we must use multiprocessing to get parallelism.
If we use multiprocessing we have to assume sometimes we cannot use  a direct `fork()` to
get a subprocess.
   
   There might be other bugs as if do not have a direct `fork()` as arguments to the children
process need to be serialized and reserialized. This has two implications:
   - Pointer equality assumptions are broken -- even if two objects shared a pointer to a
resource before they might not after serialization and reserialization. This one is annoying.
   - Things passed to subprocesses must be pickleable which they might not be. The solution
is to make things pickleable like I did here by removing the anonymous function closure.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



Mime
View raw message