tvm-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <>
Subject [GitHub] [tvm] AndrewZhaoLuo edited a comment on pull request #8363: [Tuning] Allow multiprocessing spawn to work (on macOS llvm at least)
Date Wed, 30 Jun 2021 17:47:33 GMT

AndrewZhaoLuo edited a comment on pull request #8363:

   The main lesson here is if we use python we must use multiprocessing to get parallelism.
If we use multiprocessing we have to assume sometimes we cannot use  a direct `fork()` to
get a subprocess.
   There might be other bugs as if do not have a direct `fork()` as arguments to the children
process need to be serialized and reserialized. This has two implications:
   - Pointer equality assumptions are broken -- even if two objects shared a pointer to a
resource before they might not after serialization and reserialization. This one is annoying.
   - Things passed to subprocesses must be pickleable which they might not be. The solution
is to make things pickleable like I did here by removing the anonymous function closure.

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail:

For queries about this service, please contact Infrastructure at:

View raw message