From commits-return-40890-archive-asf-public=cust-asf.ponee.io@tvm.apache.org Wed Jun 30 17:47:35 2021 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mxout1-he-de.apache.org (mxout1-he-de.apache.org [95.216.194.37]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id 2E752180670 for ; Wed, 30 Jun 2021 19:47:35 +0200 (CEST) Received: from mail.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mxout1-he-de.apache.org (ASF Mail Server at mxout1-he-de.apache.org) with SMTP id 9D55A60AEE for ; Wed, 30 Jun 2021 17:47:34 +0000 (UTC) Received: (qmail 23189 invoked by uid 500); 30 Jun 2021 17:47:33 -0000 Mailing-List: contact commits-help@tvm.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@tvm.apache.org Delivered-To: mailing list commits@tvm.apache.org Received: (qmail 23180 invoked by uid 99); 30 Jun 2021 17:47:33 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 30 Jun 2021 17:47:33 +0000 From: =?utf-8?q?GitBox?= To: commits@tvm.apache.org Subject: =?utf-8?q?=5BGitHub=5D_=5Btvm=5D_AndrewZhaoLuo_edited_a_comment_on_pull_requ?= =?utf-8?q?est_=238363=3A_=5BTuning=5D_Allow_multiprocessing_spawn_to_work_?= =?utf-8?q?=28on_macOS_llvm_at_least=29?= Message-ID: <162507525360.26479.14760125827967870973.asfpy@gitbox.apache.org> Date: Wed, 30 Jun 2021 17:47:33 -0000 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit In-Reply-To: References: AndrewZhaoLuo edited a comment on pull request #8363: URL: https://github.com/apache/tvm/pull/8363#issuecomment-871606570 The main lesson here is if we use python we must use multiprocessing to get parallelism. If we use multiprocessing we have to assume sometimes we cannot use a direct `fork()` to get a subprocess. There might be other bugs as if do not have a direct `fork()` as arguments to the children process need to be serialized and reserialized. This has two implications: - Pointer equality assumptions are broken -- even if two objects shared a pointer to a resource before they might not after serialization and reserialization. This one is annoying. - Things passed to subprocesses must be pickleable which they might not be. The solution is to make things pickleable like I did here by removing the anonymous function closure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org