arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravindra Pindikura <ravin...@dremio.com>
Subject Re: Boost and manylinux CI builds
Date Sat, 02 Mar 2019 03:58:13 GMT
Thanks Uwe.

For the record (in case someone needs to do it again), these are the steps :

1. Make the change in build_boost.sh

2. Setup an account on quay.io <http://quay.io/> and link to your GitHub account

3. In quay.io <http://quay.io/>,  Add a new repository using :

A. Link to GitHub repository push  
B. Trigger build on changes to a specific branch (eg. myquay) of the repo (eq. pravindra/arrow)
C. Set Dockerfile location to "/python/manylinux1/Dockerfile-x86_64_base”
D. Set Context location to "/python/manylinux1”

4. Push change (in step 1) to the branch specified in step 3B 

This should trigger a build in quay.io <http://quay.io/>, the build takes about 2 hrs
to finish.

5. Add a tag “latest” to the build after step 4 finishes, save the URL of the build (eg.
quay.io/pravindra <http://quay.io/pravindra/arrow_manylinux1_x86_64_base:latest>/arrow_manylinux1_x86_64_base:latest
<http://quay.io/pravindra/arrow_manylinux1_x86_64_base:latest>)

6. In your arrow PR,

- include the change from 1.
- update travis_script_manylinux.sh to point to the location from step 5.

Thanks & regards,
Ravindra.

> On Feb 27, 2019, at 3:02 PM, Uwe L. Korn <uwelk@xhochy.com> wrote:
> 
> Hello Ravindra,
> 
> simplest thing would be when you open a pull request and I can then pick this up and
push it to my personal fork. Then a new image is built on quay.io. Otherwise, you can also
activate quay.io on your fork to get the docker image to build.
> 
> Uwe
> 
> On Wed, Feb 27, 2019, at 8:41 AM, Krisztián Szűcs wrote:
>> Hi Ravindra!
>> 
>> You'll need to rebuild the docker image and change this line accordingly:
>> https://github.com/apache/arrow/blob/master/ci/travis_script_manylinux.sh#L57
>> 
>> On Wed, Feb 27, 2019 at 8:29 AM Ravindra Pindikura <ravindra@dremio.com>
>> wrote:
>> 
>>> Hi,
>>> 
>>> I added an include for boost header file in gandiva. This compiles on
>>> ubuntu/Mac/windows, but fails with the manylinux CI entry.
>>> 
>>> I’m getting a compilation failure :
>>> 
>>> https://travis-ci.org/apache/arrow/jobs/498718755 <
>>> https://travis-ci.org/apache/arrow/jobs/498718755>
>>> /arrow/cpp/src/gandiva/decimal_xlarge.cc:29:44: fatal error:
>>> boost/multiprecision/cpp_int.hpp: No such file or directory
>>> #include "boost/multiprecision/cpp_int.hpp"
>>> ^
>>> compilation terminated.
>>> 
>>> 
>>> 
>>> @xhocy and @kszucs pointed out the manylinux1 image has a very minimal
>>> boost, and doesn’t include the multi precision files. So, the script that
>>> builds boost for manylinux1 needs to be updated for this.
>>> 
>>> 
>>> 
>>> https://github.com/apache/arrow/blob/master/python/manylinux1/scripts/build_boost.sh#L38
>>> <
>>> https://github.com/apache/arrow/blob/master/python/manylinux1/scripts/build_boost.sh#L38
>>>> 
>>> 
>>> After making change, the manylinux1 build still fails with the same error
>>> :(.
>>> 
>>> https://travis-ci.org/apache/arrow/jobs/498847622
>>> 
>>> Looks like the CI run downloads a prebuilt docker image. Do I need to
>>> update the docker image ? If yes, can you please point out the instructions
>>> for this ?
>>> 
>>> Thanks & regards,
>>> Ravindra.
>> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message