From user-return-1168-archive-asf-public=cust-asf.ponee.io@arrow.apache.org Tue Apr 13 20:16:35 2021 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mxout1-ec2-va.apache.org (mxout1-ec2-va.apache.org [3.227.148.255]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id 2546C18062C for ; Tue, 13 Apr 2021 22:16:35 +0200 (CEST) Received: from mail.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mxout1-ec2-va.apache.org (ASF Mail Server at mxout1-ec2-va.apache.org) with SMTP id 55E893F2A4 for ; Tue, 13 Apr 2021 20:16:34 +0000 (UTC) Received: (qmail 46971 invoked by uid 500); 13 Apr 2021 20:16:33 -0000 Mailing-List: contact user-help@arrow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@arrow.apache.org Delivered-To: mailing list user@arrow.apache.org Received: (qmail 46961 invoked by uid 99); 13 Apr 2021 20:16:33 -0000 Received: from spamproc1-he-de.apache.org (HELO spamproc1-he-de.apache.org) (116.203.196.100) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 13 Apr 2021 20:16:33 +0000 Received: from localhost (localhost [127.0.0.1]) by spamproc1-he-de.apache.org (ASF Mail Server at spamproc1-he-de.apache.org) with ESMTP id 00C091FF469 for ; Tue, 13 Apr 2021 20:16:33 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamproc1-he-de.apache.org X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=0 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.2, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamproc1-he-de.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=xander.ai Received: from mx1-he-de.apache.org ([116.203.227.195]) by localhost (spamproc1-he-de.apache.org [116.203.196.100]) (amavisd-new, port 10024) with ESMTP id 6-b02LzGRJcL for ; Tue, 13 Apr 2021 20:16:32 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2607:f8b0:4864:20::e35; helo=mail-vs1-xe35.google.com; envelope-from=xander@xander.ai; receiver= Received: from mail-vs1-xe35.google.com (mail-vs1-xe35.google.com [IPv6:2607:f8b0:4864:20::e35]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id 0F9EC7FD86 for ; Tue, 13 Apr 2021 20:16:31 +0000 (UTC) Received: by mail-vs1-xe35.google.com with SMTP id r4so5023644vsq.6 for ; Tue, 13 Apr 2021 13:16:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xander.ai; s=google; h=mime-version:in-reply-to:from:date:message-id:references:subject:to; bh=8nOpANNHemLRk0Ae5ItXdxY/bF5eGXvHt4GgHjTjJ48=; b=Gm3QGyPBxTxxurS3Y97SLSqi2FZuXoRjhm87pweuRKkwt19KwMZgHnh6ZBSgaLsOji rUz6mnAcMVxiWM+uKuqrPwdM/1csG/HfrNMnmOjjPzeoxE9qzPKm9lnbnlcQEOhqkfo3 AzKdXLL4fMwpTDPa3UwHwbX/ONTYarSEIqz+RC3KUibu7opkFrGcE1JIwXJiPyaf8Wu/ iWYXwWHilm3GIXvKHfMcsBM5XBz/1wp0hOIqEtiPbr1W81rBPTWtKimBvp8HYMyaDfQi 0nFreS3f957IAUElY1nUG+IauF3JSIoP/oyo+WX/YTuEQCkGzLLirpFCmggvIrwy4Dxg HsTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:from:date:message-id :references:subject:to; bh=8nOpANNHemLRk0Ae5ItXdxY/bF5eGXvHt4GgHjTjJ48=; b=mNWA+B8efWhCU6x+swC7S6isJgIzn7XV9oserH+K4PQD/6qc8nb+OjNnEujy+xvhT+ Yc+Tuj+PEffZg+996RI1K8m9Z8rJD9YBWCdXL6z8t0svCNaEdW8/Rn2pijk+o2wiCYNo 4XQ60JIsZk2WM7pOMPERWoncSkom6lltNAP9+UH7HDqCXmsmrUqa24i9DzjiGVObP9Ek OphNubaq6oAF6h4/G1EwxNHipmwHow00JwReLECcH+TMxjrFy9gfunJLwjbJKsVKT1n/ QCUPlocq4zpwEyrERX667C1HbuzCiWZTHduZV1duBGTWsefXFp03ruRPI0nfjZtapMTz Mo2A== X-Gm-Message-State: AOAM533n4Hw4q4YshYjhmbLFKYrxfiJeAK6CSEtxHxJaz7CeFLTcYy+x R0WBTht6C9Uwv4hCp8kvPqr1+j+GOsrfUQ== X-Google-Smtp-Source: ABdhPJyHBPvuVjd5AbZ8/4BXkAjIzVALIj57ftl8tCoH59Xul5MRncITSHBx34SAlkZtgtpAnpqyZQ== X-Received: by 2002:a67:7745:: with SMTP id s66mr10363082vsc.3.1618344990441; Tue, 13 Apr 2021 13:16:30 -0700 (PDT) Received: from localhost (217.6.185.35.bc.googleusercontent.com. [35.185.6.217]) by smtp.gmail.com with ESMTPSA id h129sm1908999vkb.36.2021.04.13.13.16.30 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 13 Apr 2021 13:16:30 -0700 (PDT) Mime-Version: 1.0 X-Mailer: Superhuman Desktop (2021-04-12T22:05:57Z) X-Superhuman-ID: knggt4ur.31f9ec75-ab21-42b3-b573-c8a502a1a1d5 In-Reply-To: From: "Xander Dunn" Date: Tue, 13 Apr 2021 20:16:28 +0000 Message-ID: X-Superhuman-Draft-ID: draft00f8f7e4b8e2aa20 References: Subject: Re: [Python] pyarrow.gandiva unavailable on Ubuntu? To: user@arrow.apache.org, "Micah Kornfield" Content-Type: multipart/alternative; boundary=2b06ecf0b4ddba69ef981d145f5758fcd64ccbb3dbcf08eba22fdea6f144 --2b06ecf0b4ddba69ef981d145f5758fcd64ccbb3dbcf08eba22fdea6f144 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=UTF-8 Typo. The issue remains present. From my Ubuntu machine just now: ``` $ python >>> import pyarrow as pa >>> print(pa.__file__) / home/xander/anaconda3/envs/plutus_model/lib/python3.7/site-packages/pyarr= ow/__init__.py ( http://home/xander/anaconda3/envs/plutus_model/lib/python3= .7/site-packages/pyarrow/__init__.py ) >>> import pyarrow.plasma >>> import pyarrow.gandiva Traceback (most recent call last): File "", line 1, in ModuleNotFoundError: No module named 'pyarrow.gandiva' ``` The .py I'm executing on both machines is identical. Works on mac. Not foun= d on Ubuntu. On Tue, Apr 13, 2021 at 1:01 PM, Micah Kornfield < emkornfield@gmail.com > = wrote: >=20 > Hi Xander, > Was there autocorrect on this e-mail?=C2=A0 the second example shows "gon= diva" > not "gandiva" >=20 >=20 > -Micah >=20 > On Tue, Apr 13, 2021 at 12:59 PM Xander Dunn < xander@ xander. ai ( > xander@xander.ai ) > wrote: >=20 >=20 >> On my local macOS 11.2.3: >>=20 >> ``` >>=20 >> $ python --version >>=20 >> Python 3.7.10 >>=20 >> $ pip --version >>=20 >> pip 21.0.1 from >> /usr/local/anaconda3/envs/my_model/lib/python3.7/site-packages/pip (pyth= on >> 3.7) >>=20 >> $ pip list | grep pyarrow >>=20 >> pyarrow=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 3.0.0 >>=20 >> $ which python >>=20 >> /usr/local/anaconda3/envs/my_model/bin/python >>=20 >> $ python >>=20 >> >>> import pyarrow as pa >>=20 >> >>> print(pa.__file__) >>=20 >> / usr/ local/ anaconda3/ envs/ my_model/ lib/ python3. 7/ site-packages/= pyarrow/ >> __init__. py ( >> http://usr/local/anaconda3/envs/my_model/lib/python3.7/site-packages/pya= rrow/__init__.py >> ) >>=20 >> >>> import pyarrow.plasma >>=20 >> >>> import pyarrow.gandiva as ga >>=20 >> >>> print(ga.__file__) >>=20 >> / usr/ local/ anaconda3/ envs/ my_model/ lib/ python3. 7/ site-packages/= pyarrow/ >> gandiva. cpython-37m-darwin. so ( >> http://usr/local/anaconda3/envs/my_model/lib/python3.7/site-packages/pya= rrow/gandiva.cpython-37m-darwin.so >> ) >>=20 >> ``` >>=20 >>=20 >>=20 >> On my Ubuntu 14.04 instance: >>=20 >> ``` >>=20 >> $ python --version >>=20 >> Python 3.7.10 >>=20 >> $ pip --version >>=20 >> pip 21.0.1 from >> /home/xander/anaconda3/envs/my_model/lib/python3.7/site-packages/pip >> (python 3.7) >>=20 >> $ pip list | grep pyarrow >>=20 >> pyarrow=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 3.0.0 >>=20 >> $ which python >>=20 >> /home/xander/anaconda3/envs/my_model/bin/python >>=20 >> $ python >>=20 >> >>> import pyarrow as pa >>=20 >> >>> print(pa.__file__) >>=20 >> / home/ xander/ anaconda3/ envs/ my_model/ lib/ python3. 7/ site-package= s/ >> pyarrow/ __init__. py ( >> http://home/xander/anaconda3/envs/my_model/lib/python3.7/site-packages/p= yarrow/__init__.py >> ) >>=20 >> >>> import pyarrow.plasma >>=20 >> >>> import pyarrow.gondiva >>=20 >> Traceback (most recent call last): >>=20 >> File "", line 1, in >>=20 >> ModuleNotFoundError: No module named 'pyarrow.gondiva' >>=20 >> ``` >>=20 >> You can see that pyarrow.gondiva is found on mac but not on Ubuntu. Same >> Python version. Same pyarrow version. I installed both of them with `con= da >> install -c conda-forge pyarrow=3D=3D3.0.0`. >>=20 >>=20 >>=20 >> On Mac, I see the expected Cython file and library: >>=20 >> ``` >>=20 >> $ l >> /usr/local/anaconda3/envs/my_model/lib/python3.7/site-packages/pyarrow/ = | >> grep gandiva >>=20 >> -rwxrwxr-x=C2=A0=C2=A0 2 xander=C2=A0 staff=C2=A0=C2=A0 221K Apr=C2=A0 1= 12:44 gandiva. cpython-37m-darwin. >> so ( http://gandiva.cpython-37m-darwin.so/ ) >>=20 >> -rw-rw-r--=C2=A0=C2=A0 2 xander=C2=A0 staff=C2=A0=C2=A0=C2=A0 17K Jan 18= 14:00 gandiva.pyx >>=20 >> ``` >>=20 >>=20 >>=20 >> On Ubuntu, I see only the Cython file: >>=20 >> ``` >>=20 >> $ l ~/anaconda3/envs/my_model/lib/python3.7/site-packages/pyarrow/ | gre= p >> gandiva >>=20 >> -rw-rw-r-- 1 xander xander=C2=A0 17K Apr 13 12:28 gandiva.pyx >>=20 >> ``` >>=20 >>=20 >>=20 >> Is this expected? Should I be able to import pyarrow.gandiva on Ubuntu? >> Everything is run on Ubuntu so if I make use of pyarrow.gandiva I'll nee= d >> to figure out how to call it. >>=20 >>=20 >>=20 >> It's mentioned here that it was removed from Python wheels but should >> still be available in the conda install: https:/ / issues. apache. org/ = jira/ >> browse/ ARROW-10154 ( https://issues.apache.org/jira/browse/ARROW-10154 = ). >> I'm not finding it in my Ubuntu conda install. >>=20 >>=20 >>=20 >> Thanks, >>=20 >> Xander >>=20 >>=20 >>=20 >=20 > --2b06ecf0b4ddba69ef981d145f5758fcd64ccbb3dbcf08eba22fdea6f144 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=UTF-8
Typo. The issue rem= ains present. From my Ubuntu machine just now:
``= `
$ python
>>>= import pyarrow as pa
>>> print(pa.__fil= e__)
>>> import pyarrow.plasma
>>> import pyarrow.gandiva
Traceback (most recent call last):
=C2=A0 F= ile "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pyarrow.gandiva'
```

The .py I'm executing on both machines is identical. Works on mac. No= t found on Ubuntu.


On Tue, Apr 13, 20= 21 at 1:01 PM, Micah Kornfield <emkornfield@gmail.com> w= rote:
Hi Xander,
Was there autocorrect on this e-mail?=C2=A0 the second= example shows "gondiva" not "gandiva"=C2=A0

-Micah

On Tue, A= pr 13, 2021 at 12:59 PM Xander Dunn <xander@xander.ai= > wrote:
On my local macOS 11.2.3:
```
$ python --version
Python 3.7.10
$ pip --v= ersion
pip 21.0.1 from /us= r/local/anaconda3/envs/my_model/lib/python3.7/site-packages/pip (python 3.7= )
$ pip list | grep pyarro= w
pyarrow=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= 3.0.0
$ which python
=
/usr/local/anaconda3/envs/my_m= odel/bin/python
$ python
>>> import pyarrow= as pa
>>> print(= pa.__file__)
>>> import pyarrow.plasma
=
>>> import pyarrow.ga= ndiva as ga
>>> p= rint(ga.__file__)
```

On my Ubuntu 14.04 instance:
=
```
$ python --version
Python 3.7.10
$ = pip --version
pip 21.0.1 f= rom /home/xander/anaconda3/envs/my_model/lib/python3.7/site-packages/pip (p= ython 3.7)
$ pip list | gr= ep pyarrow
pyarrow=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 3.0.0
$ which py= thon
/home/xander/anaconda= 3/envs/my_model/bin/python
$ python
>>> imp= ort pyarrow as pa
>>= > print(pa.__file__)
/<= a href=3D"http://home/xander/anaconda3/envs/my_model/lib/python3.7/site-pac= kages/pyarrow/__init__.py" target=3D"_blank" rel=3D"noopener noreferrer">ho= me/xander/anaconda3/envs/my_model/lib/p= ython3.7/site-packages/pyarrow/__init__.py
>>> import pyar= row.plasma
>>> im= port pyarrow.gondiva
Trace= back (most recent call last):
=C2=A0 File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module nam= ed 'pyarrow.gondiva'
```
You can see that py= arrow.gondiva is found on mac but not on Ubuntu. Same Python version. Same = pyarrow version. I installed both of them with `conda install -c conda-forg= e pyarrow=3D=3D3.0.0`.
On Mac, I see the expected C= ython file and library:
``= `
$ l /usr/local/anaconda3= /envs/my_model/lib/python3.7/site-packages/pyarrow/ | grep gandiva
-rwxrwxr-x=C2=A0=C2=A0 2 xander=C2= =A0 staff=C2=A0=C2=A0 221K Apr=C2=A0 = 1 12:44 gandiva.cpython-37m-darwin= .so
-rw-rw-r--= =C2=A0=C2=A0 2 xander=C2=A0 staff=C2=A0=C2=A0=C2=A0 17K Jan 18 14:00 gandiva.pyx
```

On Ubuntu, I se= e only the Cython file:
``= `
$ l ~/anaconda3/envs/my_= model/lib/python3.7/site-packages/pyarrow/ | grep gandiva
-rw-rw-r-- 1 xander xander=C2=A0 17K Apr 13 12:28 gandiva.pyx
=
```

Is= this expected? Should I be able to import pyarrow.gandiva on Ubuntu? Every= thing is run on Ubuntu so if I make use of pyarrow.gandiva I'll need to= figure out how to call it.
It's mentioned here= that it was removed from Python wheels but should still be available in th= e conda install:=C2=A0https://= issues.apache.org/jira/browse/ARROW-10154= . I'm not finding it in my Ubuntu conda install.

Thanks,
Xander

--2b06ecf0b4ddba69ef981d145f5758fcd64ccbb3dbcf08eba22fdea6f144--