From user-return-291-archive-asf-public=cust-asf.ponee.io@arrow.apache.org Fri Jan 24 02:04:53 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id C228918025F for ; Fri, 24 Jan 2020 03:04:52 +0100 (CET) Received: (qmail 41626 invoked by uid 500); 24 Jan 2020 02:04:51 -0000 Mailing-List: contact user-help@arrow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@arrow.apache.org Delivered-To: mailing list user@arrow.apache.org Received: (qmail 41605 invoked by uid 99); 24 Jan 2020 02:04:51 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 24 Jan 2020 02:04:51 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id EA3311A32A0 for ; Fri, 24 Jan 2020 02:04:50 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.199 X-Spam-Level: X-Spam-Status: No, score=-0.199 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=clear-code.com Received: from mx1-he-de.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id jqaRdSlBPSUt for ; Fri, 24 Jan 2020 02:04:48 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=153.126.206.245; helo=mail.clear-code.com; envelope-from=kou@clear-code.com; receiver= Received: from mail.clear-code.com (mail.clear-code.com [153.126.206.245]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id 9B3A57E13C for ; Fri, 24 Jan 2020 02:04:46 +0000 (UTC) Received: from localhost (flh2-133-200-137-193.tky.mesh.ad.jp [133.200.137.193]) by mail.clear-code.com (Postfix) with ESMTPSA id BDBE9D50F0 for ; Fri, 24 Jan 2020 11:04:35 +0900 (JST) DKIM-Filter: OpenDKIM Filter v2.11.0 mail.clear-code.com BDBE9D50F0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=clear-code.com; s=default; t=1579831475; bh=imtInwxFXO0cJLBSmNft8Blv74iJ22ZXBmxzwYkidrc=; h=Date:To:Subject:From:In-Reply-To:References:From; b=c/fOdE++faNng/Xs2KuaPcOv7bOJhtfE9Qp5SpQivKUYavmtJp6EDR5mAKCF5hjUu 4ZF+WKmEVgkyuAMYduTEt12r+DwGpC/JWSwO9cbxE13fj2ioPghfOwCwNziMx6i60t fUJjcqczR8ZGY4io2yZ/9+MFzsn5H+EotWWV0vbY= Date: Fri, 24 Jan 2020 11:04:35 +0900 (JST) Message-Id: <20200124.110435.1826254916172559762.kou@clear-code.com> To: user@arrow.apache.org Subject: Re: Pyarrow build/install from source in ubuntu not working From: Sutou Kouhei In-Reply-To: References: X-Mailer: Mew version 6.8 on Emacs 26.3 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: BDBE9D50F0 X-Spamd-Result: default: False [1.40 / 999.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MID_CONTAINS_FROM(1.00)[]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:2518, ipnet:133.200.0.0/14, country:JP] X-Rspamd-Server: mail.clear-code.com Hi, Changing > RUN python3 -c 'import pyarrow' to RUN LD_LIBRARY_PATH=/usr/local/lib python3 -c 'import pyarrow' works on my environment. Other solution: Adding ENV LD_LIBRARY_PATH=/usr/local/lib before RUN python3 -c 'import pyarrow' Dockerfile: ... RUN bash install_arrow.sh ENV LD_LIBRARY_PATH=/usr/local/lib RUN python3 -c 'import pyarrow' Thanks, -- kou In "Pyarrow build/install from source in ubuntu not working" on Thu, 23 Jan 2020 17:38:33 -0800, Anna Waldron wrote: > Hi, > > I am trying to build and install pyarrow from source in an ubuntu 18.04 > docker image and getting the following error when attempting to import the > module: > > Traceback (most recent call last): >> File "", line 1, in >> File >> "/usr/local/lib/python3.6/dist-packages/pyarrow-0.14.0-py3.6-linux-x86_64.egg/pyarrow/__init__.py", >> line 49, in >> from pyarrow.lib import cpu_count, set_cpu_count >> ImportError: libarrow.so.14: cannot open shared object file: No such file >> or directory >> > > Here is the Dockerfile I am using: > > FROM ubuntu:18.04 >> RUN apt-get update >> RUN apt-get install -y git >> RUN mkdir /arrow >> RUN git clone https://github.com/apache/arrow.git /arrow >> WORKDIR /arrow/arrow >> RUN git checkout apache-arrow-0.14.0 >> WORKDIR / > > COPY install_arrow.sh /install_arrow.sh > > RUN bash install_arrow.sh > > >> RUN python3 -c 'import pyarrow' > > > and the install_arrow.sh script copied into the image: > > export ARROW_BUILD_TYPE=release >> export ARROW_HOME=/usr/local \ >> PARQUET_HOME=/usr/local >> export PYTHON_EXECUTABLE=/usr/bin/python3 >> > > > # install requirements >> export DEBIAN_FRONTEND="noninteractive" >> apt-get update >> apt-get install -y --no-install-recommends apt-utils >> apt-get install -y git python3-minimal python3-pip autoconf libtool >> apt-get install -y cmake \ >> python3-dev \ >> libjemalloc-dev libboost-dev \ >> build-essential \ >> libboost-filesystem-dev \ >> libboost-regex-dev \ >> libboost-system-dev \ >> flex \ >> bison >> pip3 install --no-cache-dir six pytest numpy cython > > >> mkdir -p /arrow/cpp/build \ >> && cd /arrow/cpp/build \ >> && cmake -DCMAKE_BUILD_TYPE=$ARROW_BUILD_TYPE \ >> -DOPENSSL_ROOT_DIR=/usr/local/ssl \ >> -DCMAKE_INSTALL_LIBDIR=lib \ >> -DCMAKE_INSTALL_PREFIX=$ARROW_HOME \ >> -DARROW_PARQUET=ON \ >> -DARROW_PYTHON=ON \ >> -DARROW_PLASMA=ON \ >> -DARROW_BUILD_TESTS=OFF \ >> -DPYTHON_EXECUTABLE=$PYTHON_EXECUTABLE \ >> .. \ >> && make -j$(nproc) \ >> && make install \ >> && cd /arrow/python \ >> && python3 setup.py build_ext --build-type=$ARROW_BUILD_TYPE >> --with-parquet \ >> && python3 setup.py install >> > > > LD_LIBRARY_PATH=/usr/local/lib >> > > I'm using Docker 19.03.5 on Ubuntu 18.04.3 LTS to build the image. > > Thanks in advance for any help. > > Anna