From user-return-879-archive-asf-public=cust-asf.ponee.io@arrow.apache.org Sun Jan 3 19:22:24 2021 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mxout1-ec2-va.apache.org (mxout1-ec2-va.apache.org [3.227.148.255]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id 67EE5180663 for ; Sun, 3 Jan 2021 20:22:24 +0100 (CET) Received: from mail.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mxout1-ec2-va.apache.org (ASF Mail Server at mxout1-ec2-va.apache.org) with SMTP id 9E9784400E for ; Sun, 3 Jan 2021 19:22:23 +0000 (UTC) Received: (qmail 52122 invoked by uid 500); 3 Jan 2021 19:22:23 -0000 Mailing-List: contact user-help@arrow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@arrow.apache.org Delivered-To: mailing list user@arrow.apache.org Received: (qmail 52112 invoked by uid 99); 3 Jan 2021 19:22:22 -0000 Received: from spamproc1-he-fi.apache.org (HELO spamproc1-he-fi.apache.org) (95.217.134.168) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 03 Jan 2021 19:22:22 +0000 Received: from localhost (localhost [127.0.0.1]) by spamproc1-he-fi.apache.org (ASF Mail Server at spamproc1-he-fi.apache.org) with ESMTP id B313DC03FE for ; Sun, 3 Jan 2021 19:22:21 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamproc1-he-fi.apache.org X-Spam-Flag: NO X-Spam-Score: -0.002 X-Spam-Level: X-Spam-Status: No, score=-0.002 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.2, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamproc1-he-fi.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-ec2-va.apache.org ([116.203.227.195]) by localhost (spamproc1-he-fi.apache.org [95.217.134.168]) (amavisd-new, port 10024) with ESMTP id TI82yq7T-TWc for ; Sun, 3 Jan 2021 19:22:20 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=209.85.167.47; helo=mail-lf1-f47.google.com; envelope-from=lgautier@gmail.com; receiver= Received: from mail-lf1-f47.google.com (mail-lf1-f47.google.com [209.85.167.47]) by mx1-ec2-va.apache.org (ASF Mail Server at mx1-ec2-va.apache.org) with ESMTPS id 6AD72BCC0D for ; Sun, 3 Jan 2021 19:22:20 +0000 (UTC) Received: by mail-lf1-f47.google.com with SMTP id o17so59718766lfg.4 for ; Sun, 03 Jan 2021 11:22:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=wMAvsp5J9e77FThtNzE4uMG7m29ut3eUa1YPBKScrCw=; b=AWhS3jSLEZN218BVZ2xtS5LJiZTQz1igtPUBfdDP4nkL02rKc8WSnB5OgjjvY8aZbS 8lW6et243LHCZ5gpPl+BzUKQIly17fTjEMCErU7eY7nsIYc5Igh5qouiaK1HUgPkHsth ICC+a3tjJVAdVW3lvZmZWZwqy38ooht4ZCK9az4kOfYasUxkbJZnMCqKOUNaEdq9CjUF YAcxHSW69gEEyXNg2DdEz++vGJfrrI7aZQyglsvFHBk9lUvKN4ls95IwtCVhOwosw5TZ tp5mNGnwAJQCxBXby7te4hzRwgn6gUzEw66LUoEqMNcuhiiiJHLq3kQhMjo0ivVPIOsb vjEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=wMAvsp5J9e77FThtNzE4uMG7m29ut3eUa1YPBKScrCw=; b=ndfdWMkMzDieen7yOKX+8Db3GugBgmpRdDk7XvTenCiyj/vP0fMWpe8htprz4umEWw Wb+aVdCFdc/Z617YWpBlA1/Tka+dydaRrZWHiTdXe+DzdcKDHuvEQCweFiQB+oOF+U6V AciEYYao7CXpC01q+kirQZvDLq94WaCCKaxgOBg14ThJIK7DyL31cNt9cW2KWrWUs68N EimheDGel7O2ECrnIkvehIn3oJLwkfzc27+KE78kTkTB3Xo34zMISPvlMNgLWfmoPOcI USsJFyCats5AN47NVFTltR+kjqjnK0PZY3AgHFeC2b4ah8P25xMNIEBefOO4ivwz2Upx EhPg== X-Gm-Message-State: AOAM531mhhU/4IOrSVlcSE8Ga8u8KQWn4DTEJDOul8K6CZocH73Bjb3p aZMWg+8b/be9V7cK8kPSFnGSmiU98p+SXKfB8l3Fq6XuRVI= X-Google-Smtp-Source: ABdhPJxcX2Q9B43L6MKDWBsbG1+TuMPMlR8aLAqRy7hSYCIM2OjPbEvjvEBd+rm91yiFsN+kc7ISwuCqDHbcSiSYY5I= X-Received: by 2002:a2e:b003:: with SMTP id y3mr33068061ljk.346.1609701738948; Sun, 03 Jan 2021 11:22:18 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Laurent Gautier Date: Sun, 3 Jan 2021 14:22:07 -0500 Message-ID: Subject: Re: Sharing a C++-level pointer between Python and R To: user@arrow.apache.org Content-Type: multipart/alternative; boundary="000000000000582a1c05b803e41e" --000000000000582a1c05b803e41e Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thanks. I looked at it. I think I know how it works and the amount of plumbing should be fairly minimal. Using reticulate from rpy2 is not an option because of a long-standing issue (segfault when reticulate trying to initialize Python). I am unclear about how to bridge cffi and Cython though. Is there an example of using the C-code in pyarrow from a Python C-extension? (the same C structures and functions already made accessible to Cython). Le sam. 2 janv. 2021 =C3=A0 19:20, Wes McKinney a =C3= =A9crit : > We can go R to/from Python from an R perspective with reticulate > > https://arrow.apache.org/docs/r/articles/python.html > > There are unit tests attesting to this working: > > https://github.com/apache/arrow/blob/master/r/tests/testthat/test-python.= R > > However, I don't know of anyone trying to get interop from a Python > perspective with rpy2; there may be a small amount of plumbing needed > to get it working, others may know more. > > On Sat, Jan 2, 2021 at 5:20 PM Laurent Gautier wrote= : > > > > Hi, > > > > I am looking at sharing a pointer between Python and R. For example > create an Arrow object with Python, perform initial filtering, and then > pass a shared pointer to R through rpy2 (meaning that an R6 object is > created from this pointer and R package arrow). > > > > I found in the source a file that suggest thats Python-to-R is either > planned or may be even already functional: > https://github.com/apache/arrow/blob/master/r/src/py-to-r.cpp > > However, I did not find documentation about it. > > > > Would anyone here know more about this? > > > > Best, > > > > Laurent > > > > > --000000000000582a1c05b803e41e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thanks. I looked at it. I think I know how it works a= nd the amount of plumbing should be fairly minimal.

Using reticulate from rpy2 is not an option because of a long-standing is= sue (segfault when reticulate trying to initialize Python).
<= br>
I am unclear about how to bridge cffi and Cython though.
<= /div>
Is there an example of using the C-code in pyarrow from a Python = C-extension? (the same C structures and functions already made accessible t= o Cython).

Le=C2=A0sam. 2 janv. 2021 =C3=A0=C2=A019:20, Wes McKinney &= lt;wesmckinn@gmail.com> a =C3= =A9crit=C2=A0:
W= e can go R to/from Python from an R perspective with reticulate

https://arrow.apache.org/docs/r/articles/python.= html

There are unit tests attesting to this working:

https://github.com/apache/= arrow/blob/master/r/tests/testthat/test-python.R

However, I don't know of anyone trying to get interop from a Python
perspective with rpy2; there may be a small amount of plumbing needed
to get it working, others may know more.

On Sat, Jan 2, 2021 at 5:20 PM Laurent Gautier <lgautier@gmail.com> wrote:
>
> Hi,
>
> I am looking at sharing a pointer between Python and R. For example cr= eate an Arrow object with Python, perform initial filtering, and then pass = a shared pointer to R through rpy2 (meaning that an R6 object is created fr= om this pointer and R package arrow).
>
> I found in the source a file that suggest thats Python-to-R is either = planned or may be even already functional: https://github.com/apache/arrow/blob/master/r/src/py-to-r.cpp
> However, I did not find documentation about it.
>
> Would anyone here know more about this?
>
> Best,
>
> Laurent
>
>
--000000000000582a1c05b803e41e--