From user-return-845-archive-asf-public=cust-asf.ponee.io@arrow.apache.org Fri Dec 11 19:21:29 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mxout1-ec2-va.apache.org (mxout1-ec2-va.apache.org [3.227.148.255]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id F1266180637 for ; Fri, 11 Dec 2020 20:21:28 +0100 (CET) Received: from mail.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mxout1-ec2-va.apache.org (ASF Mail Server at mxout1-ec2-va.apache.org) with SMTP id 39B6A49ADE for ; Fri, 11 Dec 2020 19:21:28 +0000 (UTC) Received: (qmail 21245 invoked by uid 500); 11 Dec 2020 19:21:27 -0000 Mailing-List: contact user-help@arrow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@arrow.apache.org Delivered-To: mailing list user@arrow.apache.org Received: (qmail 21235 invoked by uid 99); 11 Dec 2020 19:21:27 -0000 Received: from spamproc1-he-de.apache.org (HELO spamproc1-he-de.apache.org) (116.203.196.100) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 11 Dec 2020 19:21:27 +0000 Received: from localhost (localhost [127.0.0.1]) by spamproc1-he-de.apache.org (ASF Mail Server at spamproc1-he-de.apache.org) with ESMTP id 04FD51FF39A for ; Fri, 11 Dec 2020 19:21:27 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamproc1-he-de.apache.org X-Spam-Flag: NO X-Spam-Score: -0.002 X-Spam-Level: X-Spam-Status: No, score=-0.002 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.2, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamproc1-he-de.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-ec2-va.apache.org ([116.203.227.195]) by localhost (spamproc1-he-de.apache.org [116.203.196.100]) (amavisd-new, port 10024) with ESMTP id evvGMJapF5ih for ; Fri, 11 Dec 2020 19:21:26 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=209.85.219.182; helo=mail-yb1-f182.google.com; envelope-from=fernando.j.herrera@gmail.com; receiver= Received: from mail-yb1-f182.google.com (mail-yb1-f182.google.com [209.85.219.182]) by mx1-ec2-va.apache.org (ASF Mail Server at mx1-ec2-va.apache.org) with ESMTPS id 33C1EBC970 for ; Fri, 11 Dec 2020 19:21:26 +0000 (UTC) Received: by mail-yb1-f182.google.com with SMTP id l14so9039683ybq.3 for ; Fri, 11 Dec 2020 11:21:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=SvCK+75rmaK2XWEaLaCV11N5mjKpc97YAHo/Rmyn2Uc=; b=texF8BoWu9fGNqt4WJEqeyx7nkDfcCV/4yZdMqe0yOaSd+H4rv/+09ZIzS67V+1BUZ Dxb48DJyA+uMWvPNWm91amcdzWFEsAVJaLtfzWcfkNItiZef0srzaXnptA+01c+a501C /1VLZG2DUkPXEsdheWKJjitjVmFU3fSnTzIdNPteOqnFEKxso7EZIRil2Cat96/bpUnU Dsapkvk7fROYvdStNc+GhgzKkLPTAgGKBQOCPxmrwycYBt1Y0mO5wVrJhP4GYshyH/2W Zqu91ANqZaQSRWqgQBCaURf5Lfaqyn7b7ILjAx/XOonrqHeZS6w+CneLiUtmAj1TilRz lSRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=SvCK+75rmaK2XWEaLaCV11N5mjKpc97YAHo/Rmyn2Uc=; b=pOmTA9Bs+QQOnJWxBiudzbRQ0fDxz0S5bwpc193WjIxhTZdvY1yIh2eliQBLdjgkvQ vw48W+VwcdEQPQqptlU43uUiJ/LMOpRgciwH1FSStoyInWjWsy6AGBnNqqL2RRMYghuf AdvzcV6kolEf9byvXk/RoKOs7EWeHM5NZh40raMppUYeRJFTXDj7qB8RynnhR47tbXJP ZvnySb64QhFzKxjCl35Dv1ndFO5y/xojALJo6Gdb/41of87lT9wKBPfyAJMfsPzTKguE 4xHTYfdPcSYBqEiDGz1w+n+CQRmldHHUh+lk8tvFo0N00eSJktEFJrA9D9hola0oF4oX Tjyw== X-Gm-Message-State: AOAM532QpiqjVmagJPdbO1BJVlekV/OP0FDHSPnfP8cV6H99fEny9srh r+VAPCsw7sf5Zd3zHm/ulWrjpL62NDuXbD+F9E5jghip X-Google-Smtp-Source: ABdhPJx+LxBzKwF5/R7oxBr/m8r2cs4Vo+KVW+fS920ia+PpNfH6wqQVRwisbsJbx++xGtR6sJojlPgbRW0Jyiewu98= X-Received: by 2002:a25:1d46:: with SMTP id d67mr19536476ybd.480.1607714485593; Fri, 11 Dec 2020 11:21:25 -0800 (PST) MIME-Version: 1.0 From: Fernando Herrera Date: Fri, 11 Dec 2020 19:21:14 +0000 Message-ID: Subject: [python] Share data between multiple processes To: user@arrow.apache.org Content-Type: multipart/alternative; boundary="000000000000d06ff205b635322f" --000000000000d06ff205b635322f Content-Type: text/plain; charset="UTF-8" Hello, I'm implementing a text data analyzer that compares all the data against each other. This process benefits a lot from using multiprocessing. However, I'm having problems sharing the data between the processes and I think arrow will solve this easily. I was wondering if someone could point me in the right direction. How does one go sharing the location of the mapped data among the different processes spawned by the main process? Do I have to share the table pointer between the processes? Any guidance would be much appreciated Fernando --000000000000d06ff205b635322f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hello,

I'm implementing a text data= analyzer that compares all the data against each other. This=C2=A0 process= benefits a lot from using multiprocessing. However, I'm having problem= s sharing the data between the processes and I think arrow will solve this = easily.=C2=A0
I was wondering if someone could point me=C2=A0in t= he right direction. How does one go sharing the location of the mapped data= among the different processes spawned=C2=A0by the main process? Do I have = to share the table pointer between the processes?=C2=A0

Any guidance would be much appreciated
Fernando
--000000000000d06ff205b635322f--