From user-return-1276-archive-asf-public=cust-asf.ponee.io@arrow.apache.org Thu Jun 17 15:57:57 2021 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mxout1-he-de.apache.org (mxout1-he-de.apache.org [95.216.194.37]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id 27565180658 for ; Thu, 17 Jun 2021 17:57:57 +0200 (CEST) Received: from mail.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mxout1-he-de.apache.org (ASF Mail Server at mxout1-he-de.apache.org) with SMTP id 157AE61EAD for ; Thu, 17 Jun 2021 15:57:55 +0000 (UTC) Received: (qmail 6508 invoked by uid 500); 17 Jun 2021 15:57:54 -0000 Mailing-List: contact user-help@arrow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@arrow.apache.org Delivered-To: mailing list user@arrow.apache.org Received: (qmail 6498 invoked by uid 99); 17 Jun 2021 15:57:54 -0000 Received: from spamproc1-he-de.apache.org (HELO spamproc1-he-de.apache.org) (116.203.196.100) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 17 Jun 2021 15:57:54 +0000 Received: from localhost (localhost [127.0.0.1]) by spamproc1-he-de.apache.org (ASF Mail Server at spamproc1-he-de.apache.org) with ESMTP id 2151F1FF491 for ; Thu, 17 Jun 2021 15:57:54 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamproc1-he-de.apache.org X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=0 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.2, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamproc1-he-de.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=ucsc.edu Received: from mx1-he-de.apache.org ([116.203.227.195]) by localhost (spamproc1-he-de.apache.org [116.203.196.100]) (amavisd-new, port 10024) with ESMTP id zPjSSTxjGnwa for ; Thu, 17 Jun 2021 15:57:52 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2a00:1450:4864:20::530; helo=mail-ed1-x530.google.com; envelope-from=akmontan@ucsc.edu; receiver= Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com [IPv6:2a00:1450:4864:20::530]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id AC5B47FFD5 for ; Thu, 17 Jun 2021 15:57:52 +0000 (UTC) Received: by mail-ed1-x530.google.com with SMTP id c7so3791422edn.6 for ; Thu, 17 Jun 2021 08:57:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ucsc.edu; s=ucsc-google-2018; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=bbs87fR32V01Az6551vVfZ7swpDwFCXRaa8JlAOh9u4=; b=PRNfyIGc+8iYnWwKJpDDD2s/emaFAmZYlyjXRIPWpc5/b1ZvAo4RB+RbH6FHs0e1a+ UXH361yOSg9vLoBjwdehlEVY6XnN1xLdbDlaLNC52hgBMStJQpEtwwOhYXaJFBU8iWlq Afh0gZUy9kVwAM7ywFlnidEijbORJhXdYsZw32grlYRKO/TbnVzRxkwUAHUzvmLhECto /XJ1GbvBpPyRTYfX17EDy7VxcwcZySUGYj1yy6eIVTva2kOtVGm0N+T1g52+G82JDXzr /1Dt/g/NxgxTcySOvu73dLS/PiVma1bmkOS5pZmmTvH/F8z21XAUBN/nwoXRaBS3BxSF 269A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=bbs87fR32V01Az6551vVfZ7swpDwFCXRaa8JlAOh9u4=; b=f4VdoFiNt5F+2su/sUKR+0Y7ZT1Fn7BG25EFtIk25n9z8IyhDtYthJMsL0wdmXmxLQ InTQPwuoEncdZYgAT5TprdHs7FKqgU0CdraY1Hwu1lPN3FrE3kJSzstTiYvGT2XRg2Ak KnU+gXzR0vsGSkb5Hh8DxKc/HuzQy3CZds4tcgZ1eHQaVoH5lTpSNPj2lNncoZNKCCdN Nd6SyzK9OpXPS69LT7MdlTcfSf0zrTZXHd+0wec+92V9X9Q2PYvEGwkS9fZ9UbRgOTL4 Cap3/7JkzKK1t0tDdmMTnO69tUnVa2qMpl+MrwTWjAhITFFUG76Tt8pwHp4MXYZcHK/w FutQ== X-Gm-Message-State: AOAM532JWEkOLLn0Etox1mF+8f0EcEQp78UGKrYlMjOCFzmo1KsP8dNs ZYDVLaRjIoUM3y3Ide7Be00Qjgjr4kAMwz1EWpBTd+1TdXw5rg== X-Google-Smtp-Source: ABdhPJzKsCwWyXEeSRz7b60G6/MNTaHFa3CC9YzskFXuHov9m0KXIGG+0vY/n0YbUvaKqy1aailrWMD1yLVC1qHDO3k= X-Received: by 2002:a05:6402:4404:: with SMTP id y4mr7699119eda.282.1623945471528; Thu, 17 Jun 2021 08:57:51 -0700 (PDT) MIME-Version: 1.0 References: <20210608091413.1e2fe752@fsol> In-Reply-To: From: Aldrin Date: Thu, 17 Jun 2021 08:57:40 -0700 Message-ID: Subject: Re: Understanding R and C++ using cpp11 To: user@arrow.apache.org Content-Type: multipart/alternative; boundary="000000000000f74bc305c4f844d8" --000000000000f74bc305c4f844d8 Content-Type: text/plain; charset="UTF-8" An update: I have organized my code like an R package (after reading more about NAMESPACE file), and I used `devtools::document()` to generate source files (per cpp11 docs ). Unfortunately, the generated source does not propagate my include directives, which seems to make clear why using `cpp_source` does not work for my code. Here is a code excerpt of what is generated: // skytethr.cpp arrow::Result R_GetReaderForKey(cpp11::r_string table_key); extern "C" SEXP _skytethr_R_GetReaderForKey(SEXP table_key) { BEGIN_CPP11 return cpp11::as_sexp(R_GetReaderForKey(cpp11::as_cpp>(table_key))); END_CPP11 } I manually added the include of arrow to the generated source to see if that is my only problem, but I get many errors related to the return value being wrapped in `cpp11::as_sexp`. To continue my proof-of-concept, I can change my C++ code to return a simpler type (just an arrow table, maybe), but I think I ultimately will need to figure out how to return a type that is defined by the `arrow` package. Other than doing a short-term copy-paste of headers, is there a recommended path forward on how to return a type that R knows about from the `arrow` package? My first thought is that maybe I can import some variables from the arrow namespace in my namespace file , maybe in the essence of this example ? Thanks for any help or feedback! Aldrin Montana Computer Science PhD Student UC Santa Cruz On Tue, Jun 8, 2021 at 8:54 AM Aldrin wrote: > Thank you Neal! I'll check those out and I appreciate the tips! > > Hi Antoine, I do include those. That's a reason I am especially confused, > since I don't get any issue on including them. Although, interestingly, the > error " 'arrow' does not name a type " is there whether or not I include > the arrow headers. If they are not included though, compilation stops at > that step. > > Here's a gist to make it easier to see the difference. I've also included > more code just for completeness. If there's anything else I can include > that would be helpful, please let me know! > > https://gist.github.com/drin/d9f4400670eae69f5c367b37ca8c1b24 > > > > Aldrin Montana > Computer Science PhD Student > UC Santa Cruz > > > On Tue, Jun 8, 2021 at 12:14 AM Antoine Pitrou wrote: > >> On Fri, 4 Jun 2021 17:26:03 -0700 >> Aldrin wrote: >> > Hello! >> > >> > I am trying to wrap some C++ code I have that uses arrow with R to >> enable >> > users to continue working in R. Unfortunately, I get a compilation >> error of >> > `arrow does not name a type` when it gets to the return type of my >> function >> > which is >> > `arrow::Result>`. I >> > posted in RStudio in case this is purely a cpp11 library issue >> happening ( >> > https://community.rstudio.com/t/cpp11-does-not-name-a-type/106740). >> >> Did you include the Arrow headers? For example `#include "arrow/api.h"`? >> >> >> >> --000000000000f74bc305c4f844d8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
An update:

I have organized = my code like an R package (after reading more about NAMESPACE file), and I = used `devtools::document()` to generate source files (per cpp11 docs). Unfortunate= ly, the generated source does not propagate my include directives, which se= ems to make clear why using `cpp_source` does not work for my code. Here is= a code excerpt of what is generated:
=
// skytethr.cpp
arrow::Result&= lt;BatchStreamRdrPtr> R_GetReaderForKey(cpp11::r_string table_key);
e= xtern "C" SEXP _skytethr_R_GetReaderForKey(SEXP table_key) {
= =C2=A0 BEGIN_CPP11
=C2=A0 =C2=A0 return cpp11::as_sexp(R_GetReaderForKey= (cpp11::as_cpp<cpp11::decay_t<cpp11::r_string>>(table_key)));=C2=A0 END_CPP11
}

I manually added the include of arrow to the generated source to see if th= at is my only problem, but I get many errors related to the return value be= ing wrapped in `cpp11::as_sexp`.

To continue my pr= oof-of-concept, I can change my C++ code to return a simpler type (just an = arrow table, maybe), but I think I ultimately will need to figure out how t= o return a type that is defined by the `arrow` package. Other than doing a = short-term copy-paste of headers, is there a recommended path forward on ho= w to return a type that R knows about from the `arrow` package? My first th= ought is that maybe I can import some variable= s from the arrow namespace in my namespace file, maybe in the essence o= f this example?

Thanks for any help o= r feedback!

Ald= rin Montana
Computer Science PhD Student
UC Santa Cruz

<= /div>
O= n Tue, Jun 8, 2021 at 8:54 AM Aldrin <akmontan@ucsc.edu> wrote:
Thank you Neal! I'll check th= ose out and I appreciate the tips!

Hi Antoine, I d= o include those. That's a reason I am especially confused, since I don&= #39;t get any issue on including them. Although, interestingly, the error &= quot; 'arrow' does not name a type " is there whether or not I= include the arrow headers. If they are not included though, compilation st= ops at that step.

Here's a gist to make it eas= ier to see the difference. I've also included more code just for comple= teness. If there's anything else I can include that would be helpful, p= lease let me know!

=
On Tue= , Jun 8, 2021 at 12:14 AM Antoine Pitrou <antoine@python.org> wrote:
On Fri, 4 Jun 2021 17:26:03 -07= 00
Aldrin <akmontan@= ucsc.edu> wrote:
> Hello!
>
> I am trying to wrap some C++ code I have that uses arrow with R to ena= ble
> users to continue working in R. Unfortunately, I get a compilation err= or of
> `arrow does not name a type` when it gets to the return type of my fun= ction
> which is
> `arrow::Result<std::shared_ptr<arrow::ipc::RecordBatchStreamRead= er>>`. I
> posted in RStudio in case this is purely a cpp11 library issue happeni= ng (
> https://community.rstudio.com/= t/cpp11-does-not-name-a-type/106740).

Did you include the Arrow headers? For example `#include "arrow/api.h&= quot;`?



--000000000000f74bc305c4f844d8--