arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xander Dunn" <xan...@xander.ai>
Subject Re: [Python] pyarrow.gandiva unavailable on Ubuntu?
Date Tue, 13 Apr 2021 20:45:19 GMT
```

$ conda create -n fresh_env python=3.7

$ conda activate fresh_env

$ conda install -c conda-forge pyarrow==3.0.0

$ python

>>> import pyarrow as pa

>>> pa.__version__

'3.0.0'

>>> pa.__file__

'/ home/xander/anaconda3/envs/fresh_env/lib/python3.7/site-packages/pyarrow/__init__.py (
http://home/xander/anaconda3/envs/fresh_env/lib/python3.7/site-packages/pyarrow/__init__.py
) '

>>> import pyarrow.plasma

>>> import pyarrow.gandiva

Traceback (most recent call last):

File "<stdin>", line 1, in <module>

ModuleNotFoundError: No module named 'pyarrow.gandiva'

```

Same result in a fresh conda environment. The library is also still missing from the installed
location:

```

$ l /home/xander/anaconda3/envs/fresh_env/lib/python3.7/site-packages/pyarrow/ | grep gandiva

-rw-rw-r-- 3 xander xander  17K Jan 18 14:00 gandiva.pyx

```

And a correction to what I wrote earlier, I'm on Ubuntu 18.04:

```

$ lsb_release -a

No LSB modules are available.

Distributor ID: Ubuntu

Description:    Ubuntu 18.04.5 LTS

Release:        18.04

Codename:       bionic

$ conda --version

conda 4.10.0

```

I do have several arrow packages installed via apt, but I don't think that's interfering:

```

apache-arrow-archive-keyring/bionic,now 3.0.0-1 all [installed]

gir1.2-arrow-1.0/bionic,now 3.0.0-1 amd64 [installed,automatic]

libarrow-dev/bionic,now 3.0.0-1 amd64 [installed]

libarrow-glib-dev/bionic,now 3.0.0-1 amd64 [installed,automatic]

libarrow-glib300/bionic,now 3.0.0-1 amd64 [installed,automatic]

libarrow300/bionic,now 3.0.0-1 amd64 [installed]

```

Thanks for the help,

Xander

On Tue, Apr 13, 2021 at 1:25 PM, Wes McKinney < wesmckinn@gmail.com > wrote:

> 
> It looks to me like you have the wheel installed, not the conda package.
> Can you reproduce this on Ubuntu from a fresh conda environment?
> 
> On Tue, Apr 13, 2021 at 3:16 PM Xander Dunn < xander@ xander. ai (
> xander@xander.ai ) > wrote:
> 
> 
>> Typo. The issue remains present. From my Ubuntu machine just now:
>> 
>> ```
>> 
>> $ python
>> 
>> >>> import pyarrow as pa
>> 
>> >>> print(pa.__file__)
>> 
>> / home/ xander/ anaconda3/ envs/ plutus_model/ lib/ python3. 7/ site-packages/
>> pyarrow/ __init__. py (
>> http://home/xander/anaconda3/envs/plutus_model/lib/python3.7/site-packages/pyarrow/__init__.py
>> )
>> 
>> >>> import pyarrow.plasma
>> 
>> >>> import pyarrow.gandiva
>> 
>> Traceback (most recent call last):
>> 
>> File "<stdin>", line 1, in <module>
>> 
>> ModuleNotFoundError: No module named 'pyarrow.gandiva'
>> 
>> ```
>> 
>> 
>> 
>> The .py I'm executing on both machines is identical. Works on mac. Not
>> found on Ubuntu.
>> 
>> 
>> 
>> 
>> On Tue, Apr 13, 2021 at 1:01 PM, Micah Kornfield < emkornfield@ gmail. com
>> ( emkornfield@gmail.com ) > wrote:
>> 
>>> Hi Xander,
>>> Was there autocorrect on this e-mail?  the second example shows "gondiva"
>>> not "gandiva"
>>> 
>>> 
>>> -Micah
>>> 
>>> On Tue, Apr 13, 2021 at 12:59 PM Xander Dunn < xander@ xander. ai (
>>> xander@xander.ai ) > wrote:
>>> 
>>> 
>>>> On my local macOS 11.2.3:
>>>> 
>>>> ```
>>>> 
>>>> $ python --version
>>>> 
>>>> Python 3.7.10
>>>> 
>>>> $ pip --version
>>>> 
>>>> pip 21.0.1 from
>>>> /usr/local/anaconda3/envs/my_model/lib/python3.7/site-packages/pip (python
>>>> 3.7)
>>>> 
>>>> $ pip list | grep pyarrow
>>>> 
>>>> pyarrow                3.0.0
>>>> 
>>>> $ which python
>>>> 
>>>> /usr/local/anaconda3/envs/my_model/bin/python
>>>> 
>>>> $ python
>>>> 
>>>> >>> import pyarrow as pa
>>>> 
>>>> >>> print(pa.__file__)
>>>> 
>>>> / usr/ local/ anaconda3/ envs/ my_model/ lib/ python3. 7/ site-packages/
pyarrow/
>>>> __init__. py (
>>>> http://usr/local/anaconda3/envs/my_model/lib/python3.7/site-packages/pyarrow/__init__.py
>>>> )
>>>> 
>>>> >>> import pyarrow.plasma
>>>> 
>>>> >>> import pyarrow.gandiva as ga
>>>> 
>>>> >>> print(ga.__file__)
>>>> 
>>>> / usr/ local/ anaconda3/ envs/ my_model/ lib/ python3. 7/ site-packages/
pyarrow/
>>>> gandiva. cpython-37m-darwin. so (
>>>> http://usr/local/anaconda3/envs/my_model/lib/python3.7/site-packages/pyarrow/gandiva.cpython-37m-darwin.so
>>>> )
>>>> 
>>>> ```
>>>> 
>>>> 
>>>> 
>>>> On my Ubuntu 14.04 instance:
>>>> 
>>>> ```
>>>> 
>>>> $ python --version
>>>> 
>>>> Python 3.7.10
>>>> 
>>>> $ pip --version
>>>> 
>>>> pip 21.0.1 from
>>>> /home/xander/anaconda3/envs/my_model/lib/python3.7/site-packages/pip
>>>> (python 3.7)
>>>> 
>>>> $ pip list | grep pyarrow
>>>> 
>>>> pyarrow                3.0.0
>>>> 
>>>> $ which python
>>>> 
>>>> /home/xander/anaconda3/envs/my_model/bin/python
>>>> 
>>>> $ python
>>>> 
>>>> >>> import pyarrow as pa
>>>> 
>>>> >>> print(pa.__file__)
>>>> 
>>>> / home/ xander/ anaconda3/ envs/ my_model/ lib/ python3. 7/ site-packages/
>>>> pyarrow/ __init__. py (
>>>> http://home/xander/anaconda3/envs/my_model/lib/python3.7/site-packages/pyarrow/__init__.py
>>>> )
>>>> 
>>>> >>> import pyarrow.plasma
>>>> 
>>>> >>> import pyarrow.gondiva
>>>> 
>>>> Traceback (most recent call last):
>>>> 
>>>> File "<stdin>", line 1, in <module>
>>>> 
>>>> ModuleNotFoundError: No module named 'pyarrow.gondiva'
>>>> 
>>>> ```
>>>> 
>>>> You can see that pyarrow.gondiva is found on mac but not on Ubuntu. Same
>>>> Python version. Same pyarrow version. I installed both of them with `conda
>>>> install -c conda-forge pyarrow==3.0.0`.
>>>> 
>>>> 
>>>> 
>>>> On Mac, I see the expected Cython file and library:
>>>> 
>>>> ```
>>>> 
>>>> $ l
>>>> /usr/local/anaconda3/envs/my_model/lib/python3.7/site-packages/pyarrow/ |
>>>> grep gandiva
>>>> 
>>>> -rwxrwxr-x   2 xander  staff   221K Apr  1 12:44 gandiva. cpython-37m-darwin.
>>>> so ( http://gandiva.cpython-37m-darwin.so/ )
>>>> 
>>>> -rw-rw-r--   2 xander  staff    17K Jan 18 14:00 gandiva.pyx
>>>> 
>>>> ```
>>>> 
>>>> 
>>>> 
>>>> On Ubuntu, I see only the Cython file:
>>>> 
>>>> ```
>>>> 
>>>> $ l ~/anaconda3/envs/my_model/lib/python3.7/site-packages/pyarrow/ | grep
>>>> gandiva
>>>> 
>>>> -rw-rw-r-- 1 xander xander  17K Apr 13 12:28 gandiva.pyx
>>>> 
>>>> ```
>>>> 
>>>> 
>>>> 
>>>> Is this expected? Should I be able to import pyarrow.gandiva on Ubuntu?
>>>> Everything is run on Ubuntu so if I make use of pyarrow.gandiva I'll need
>>>> to figure out how to call it.
>>>> 
>>>> 
>>>> 
>>>> It's mentioned here that it was removed from Python wheels but should
>>>> still be available in the conda install: https:/ / issues. apache. org/ jira/
>>>> browse/ ARROW-10154 ( https://issues.apache.org/jira/browse/ARROW-10154 ).
>>>> I'm not finding it in my Ubuntu conda install.
>>>> 
>>>> 
>>>> 
>>>> Thanks,
>>>> 
>>>> Xander
>>>> 
>>> 
>>> 
>> 
>> 
> 
>
Mime
View raw message