From user-return-1167-archive-asf-public=cust-asf.ponee.io@arrow.apache.org Tue Apr 13 20:02:18 2021 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mxout1-ec2-va.apache.org (mxout1-ec2-va.apache.org [3.227.148.255]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id 1444218062C for ; Tue, 13 Apr 2021 22:02:18 +0200 (CEST) Received: from mail.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mxout1-ec2-va.apache.org (ASF Mail Server at mxout1-ec2-va.apache.org) with SMTP id 488093F3A7 for ; Tue, 13 Apr 2021 20:02:17 +0000 (UTC) Received: (qmail 20992 invoked by uid 500); 13 Apr 2021 20:02:16 -0000 Mailing-List: contact user-help@arrow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@arrow.apache.org Delivered-To: mailing list user@arrow.apache.org Received: (qmail 20981 invoked by uid 99); 13 Apr 2021 20:02:16 -0000 Received: from spamproc1-he-fi.apache.org (HELO spamproc1-he-fi.apache.org) (95.217.134.168) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 13 Apr 2021 20:02:16 +0000 Received: from localhost (localhost [127.0.0.1]) by spamproc1-he-fi.apache.org (ASF Mail Server at spamproc1-he-fi.apache.org) with ESMTP id F0EE5C047B for ; Tue, 13 Apr 2021 20:02:15 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamproc1-he-fi.apache.org X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=0 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.2, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamproc1-he-fi.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-he-de.apache.org ([116.203.227.195]) by localhost (spamproc1-he-fi.apache.org [95.217.134.168]) (amavisd-new, port 10024) with ESMTP id XkqZfu9WxmXY for ; Tue, 13 Apr 2021 20:02:12 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2a00:1450:4864:20::62a; helo=mail-ej1-x62a.google.com; envelope-from=emkornfield@gmail.com; receiver= Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id D51777FD86 for ; Tue, 13 Apr 2021 20:02:11 +0000 (UTC) Received: by mail-ej1-x62a.google.com with SMTP id u17so27919402ejk.2 for ; Tue, 13 Apr 2021 13:02:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:reply-to:from:date:message-id :subject:to; bh=9TickJA57IaXhIom4rS2RRPYqIzNhXZWLwQZH4Dmqno=; b=LH+2nb+ZREabNh/Bi0kfYTic+QYa1dYWFMzVwvyndcrL0FOpMaPUdvA97Z0sZh3fGu U7B4ybbIozrkjSFeINTLP1rJ4QXhxZXe+HR9d109ieYZb813W3keQad6MctZkLsBzLY6 hyKb9M9eeQdiL6Sb88cvRVePXfqk70qegRvJn+VUjpYrl3V8MrldMxuF30XaF9V6GHkH weWFCERQNvovQNXJrwBD4zFgXWupvyc4qa1r3nWKTparZFxWfkRcYFJkoY/C5pTKwSOV 4kI30dnXwzQSL2L1L5IhPOhx2dXTK7qQwUIUpfOlrDEI3SyAhqr1PG/yNI3qpzrfAogf sQOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to; bh=9TickJA57IaXhIom4rS2RRPYqIzNhXZWLwQZH4Dmqno=; b=drY3YiAbxM3yKzmMG4GIFvTh28/5kWhSgw8LnoxPPinIdt/gIQrpaANzdwtxutksxP ReGciIFKOAnQh3tgH6nRg/dLJvj3jCQ0HUW9a9JzzCh7ChH5/nbgn2D6u+B0HjmbB6Hd ukuiP3mFZXJ5Um9iVwZeexg9v3gqyenXabaQv+Jejrke3mJAVm2fALjguIz2Y0GZiHGV VYKNcUG4415UpShzCASlN+8lsv/uKg+8fRIpsv6j/JV3Rxj8flMoHH0iOsX/IePDpQ9V k6wtS98DoXoZB7awCJ+xxisR8+mHQ3PP9k/Dv6NU+vpnPzk+Mj2MC+88syMDc78yUufW eQ4g== X-Gm-Message-State: AOAM532Ash77bhfErFWrZXZffbcfy+KyhXK9oW/y/UGAwQHJ3jr9FKdj fSpgpG77s5KwOb/MHeBQk3xf2b7IpYjyXpdV0KJHlX/oteWTzg== X-Google-Smtp-Source: ABdhPJxMLBKVJ4PnqWDKTvMgJqvuVPYmwIxDPl/djmQNfTb6vMbgbeMVz0RjSSKNnA1Ak8BuEeETROkc3cW8Cf7O9V8= X-Received: by 2002:a17:906:a103:: with SMTP id t3mr34561125ejy.334.1618344130979; Tue, 13 Apr 2021 13:02:10 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: Reply-To: emkornfield@gmail.com From: Micah Kornfield Date: Tue, 13 Apr 2021 13:01:59 -0700 Message-ID: Subject: Re: [Python] pyarrow.gandiva unavailable on Ubuntu? To: user@arrow.apache.org Content-Type: multipart/alternative; boundary="0000000000000d2b5105bfe01bbf" --0000000000000d2b5105bfe01bbf Content-Type: text/plain; charset="UTF-8" Hi Xander, Was there autocorrect on this e-mail? the second example shows "gondiva" not "gandiva" -Micah On Tue, Apr 13, 2021 at 12:59 PM Xander Dunn wrote: > On my local macOS 11.2.3: > ``` > $ python --version > Python 3.7.10 > $ pip --version > pip 21.0.1 from > /usr/local/anaconda3/envs/my_model/lib/python3.7/site-packages/pip (python > 3.7) > $ pip list | grep pyarrow > pyarrow 3.0.0 > $ which python > /usr/local/anaconda3/envs/my_model/bin/python > $ python > >>> import pyarrow as pa > >>> print(pa.__file__) > / > usr/local/anaconda3/envs/my_model/lib/python3.7/site-packages/pyarrow/__init__.py > >>> import pyarrow.plasma > >>> import pyarrow.gandiva as ga > >>> print(ga.__file__) > / > usr/local/anaconda3/envs/my_model/lib/python3.7/site-packages/pyarrow/gandiva.cpython-37m-darwin.so > ``` > > On my Ubuntu 14.04 instance: > ``` > $ python --version > Python 3.7.10 > $ pip --version > pip 21.0.1 from > /home/xander/anaconda3/envs/my_model/lib/python3.7/site-packages/pip > (python 3.7) > $ pip list | grep pyarrow > pyarrow 3.0.0 > $ which python > /home/xander/anaconda3/envs/my_model/bin/python > $ python > >>> import pyarrow as pa > >>> print(pa.__file__) > / > home/xander/anaconda3/envs/my_model/lib/python3.7/site-packages/pyarrow/__init__.py > >>> import pyarrow.plasma > >>> import pyarrow.gondiva > Traceback (most recent call last): > File "", line 1, in > ModuleNotFoundError: No module named 'pyarrow.gondiva' > ``` > You can see that pyarrow.gondiva is found on mac but not on Ubuntu. Same > Python version. Same pyarrow version. I installed both of them with `conda > install -c conda-forge pyarrow==3.0.0`. > > On Mac, I see the expected Cython file and library: > ``` > $ l > /usr/local/anaconda3/envs/my_model/lib/python3.7/site-packages/pyarrow/ | > grep gandiva > -rwxrwxr-x 2 xander staff 221K Apr 1 12:44 > gandiva.cpython-37m-darwin.so > -rw-rw-r-- 2 xander staff 17K Jan 18 14:00 gandiva.pyx > ``` > > On Ubuntu, I see only the Cython file: > ``` > $ l ~/anaconda3/envs/my_model/lib/python3.7/site-packages/pyarrow/ | grep > gandiva > -rw-rw-r-- 1 xander xander 17K Apr 13 12:28 gandiva.pyx > ``` > > Is this expected? Should I be able to import pyarrow.gandiva on Ubuntu? > Everything is run on Ubuntu so if I make use of pyarrow.gandiva I'll need > to figure out how to call it. > > It's mentioned here that it was removed from Python wheels but should > still be available in the conda install: > https://issues.apache.org/jira/browse/ARROW-10154. I'm not finding it in > my Ubuntu conda install. > > Thanks, > Xander > > --0000000000000d2b5105bfe01bbf Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Xander,
Was there autocorrect on this e-mail?=C2=A0= the second example shows "gondiva" not "gandiva"=C2=A0=

-Micah

=
On Tue, Apr 13, 2021 at 12:59 PM Xand= er Dunn <xander@xander.ai> wr= ote:
<= div>
On my local macOS 11.2.3:
```
$ python= --version
Python 3.7.10
$ pip --version
pip 21.0.1 from /usr/local/anaconda3/envs/my_model/lib/python3.7/si= te-packages/pip (python 3.7)
$ pip list | grep pyarrow
pyarrow=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 3.0.0
$ which python
=
/usr/local/anaconda3/envs/my_model/bin/python
$ python
>>> import pyarrow as pa
>>> p= rint(pa.__file__)
>>> import pyarrow.plasma
>>> import pyarrow.gandiva as ga
>>> print= (ga.__file__)
```

On my Ubuntu 14.04 instance:
```
$ python --version
Python 3.7.10
$ pip --ver= sion
pip 21.0.1 from /home/xander/anaconda3/envs/my_model/lib= /python3.7/site-packages/pip (python 3.7)
$ pip list | grep p= yarrow
pyarrow=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 3.0.0
$ which p= ython
/home/xander/anaconda3/envs/my_model/bin/python
$ python
>>> import pyarrow as pa
>>> print(pa.__file__)
>>> import pyarrow= .plasma
>>> import pyarrow.gondiva
Tra= ceback (most recent call last):
=C2=A0 File "<stdin&g= t;", line 1, in <module>
ModuleNotFoundError: No m= odule named 'pyarrow.gondiva'
```
You c= an see that pyarrow.gondiva is found on mac but not on Ubuntu. Same Python = version. Same pyarrow version. I installed both of them with `conda install= -c conda-forge pyarrow=3D=3D3.0.0`.

On Mac, I= see the expected Cython file and library:
```
= $ l /usr/local/anaconda3/envs/my_model/lib/python3.7/site-packages/pyarrow/= | grep gandiva
-rwxrwxr-x=C2=A0=C2=A0 2 xander=C2=A0 staff= =C2=A0=C2=A0 221K Apr=C2=A0 1 12:44 gandiva.cpython-37m-darwin.so=
-rw-rw-r--=C2=A0=C2=A0 2 xander=C2=A0 staff=C2=A0=C2=A0=C2= =A0 17K Jan 18 14:00 gandiva.pyx
```

On Ubuntu, I see only the Cython file:
```<= br>
$ l ~/anaconda3/envs/my_model/lib/python3.7/site-packages/pya= rrow/ | grep gandiva
-rw-rw-r-- 1 xander xander=C2=A0 17K Apr 13 12:28 gandiva.pyx
```

Is this expected? Should I be able to import pyarrow.gandiva on Ubun= tu? Everything is run on Ubuntu so if I make use of pyarrow.gandiva I'l= l need to figure out how to call it.

It's = mentioned here that it was removed from Python wheels but should still be a= vailable in the conda install:=C2=A0https://issues.apache.org/jira/bro= wse/ARROW-10154. I'm not finding it in my Ubuntu conda install.
=

Thanks,
Xander
=
3D""

= --0000000000000d2b5105bfe01bbf--