arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravindra Pindikura <ravin...@dremio.com>
Subject Re: Boost and manylinux CI builds
Date Mon, 04 Mar 2019 08:34:58 GMT


> On Mar 4, 2019, at 4:45 AM, Wes McKinney <wesmckinn@gmail.com> wrote:
> 
> hi Ravindra,
> 
> Can we document this (just by copy-pasting what you wrote) on the wiki
> or someplace for future work that may touch the manylinux package
> builds? This might be a bit more discoverable than going through the
> email logs

Sure Wes. Will do via ARROW-4756

> 
> Thanks!
> Wes
> 
> On Fri, Mar 1, 2019 at 9:58 PM Ravindra Pindikura <ravindra@dremio.com> wrote:
>> 
>> Thanks Uwe.
>> 
>> For the record (in case someone needs to do it again), these are the steps :
>> 
>> 1. Make the change in build_boost.sh
>> 
>> 2. Setup an account on quay.io <http://quay.io/> and link to your GitHub account
>> 
>> 3. In quay.io <http://quay.io/>,  Add a new repository using :
>> 
>> A. Link to GitHub repository push
>> B. Trigger build on changes to a specific branch (eg. myquay) of the repo (eq. pravindra/arrow)
>> C. Set Dockerfile location to "/python/manylinux1/Dockerfile-x86_64_base”
>> D. Set Context location to "/python/manylinux1”
>> 
>> 4. Push change (in step 1) to the branch specified in step 3B
>> 
>> This should trigger a build in quay.io <http://quay.io/>, the build takes about
2 hrs to finish.
>> 
>> 5. Add a tag “latest” to the build after step 4 finishes, save the URL of the
build (eg. quay.io/pravindra <http://quay.io/pravindra/arrow_manylinux1_x86_64_base:latest>/arrow_manylinux1_x86_64_base:latest
<http://quay.io/pravindra/arrow_manylinux1_x86_64_base:latest>)
>> 
>> 6. In your arrow PR,
>> 
>> - include the change from 1.
>> - update travis_script_manylinux.sh to point to the location from step 5.
>> 
>> Thanks & regards,
>> Ravindra.
>> 
>>> On Feb 27, 2019, at 3:02 PM, Uwe L. Korn <uwelk@xhochy.com> wrote:
>>> 
>>> Hello Ravindra,
>>> 
>>> simplest thing would be when you open a pull request and I can then pick this
up and push it to my personal fork. Then a new image is built on quay.io. Otherwise, you can
also activate quay.io on your fork to get the docker image to build.
>>> 
>>> Uwe
>>> 
>>> On Wed, Feb 27, 2019, at 8:41 AM, Krisztián Szűcs wrote:
>>>> Hi Ravindra!
>>>> 
>>>> You'll need to rebuild the docker image and change this line accordingly:
>>>> https://github.com/apache/arrow/blob/master/ci/travis_script_manylinux.sh#L57
>>>> 
>>>> On Wed, Feb 27, 2019 at 8:29 AM Ravindra Pindikura <ravindra@dremio.com>
>>>> wrote:
>>>> 
>>>>> Hi,
>>>>> 
>>>>> I added an include for boost header file in gandiva. This compiles on
>>>>> ubuntu/Mac/windows, but fails with the manylinux CI entry.
>>>>> 
>>>>> I’m getting a compilation failure :
>>>>> 
>>>>> https://travis-ci.org/apache/arrow/jobs/498718755 <
>>>>> https://travis-ci.org/apache/arrow/jobs/498718755>
>>>>> /arrow/cpp/src/gandiva/decimal_xlarge.cc:29:44: fatal error:
>>>>> boost/multiprecision/cpp_int.hpp: No such file or directory
>>>>> #include "boost/multiprecision/cpp_int.hpp"
>>>>> ^
>>>>> compilation terminated.
>>>>> 
>>>>> 
>>>>> 
>>>>> @xhocy and @kszucs pointed out the manylinux1 image has a very minimal
>>>>> boost, and doesn’t include the multi precision files. So, the script
that
>>>>> builds boost for manylinux1 needs to be updated for this.
>>>>> 
>>>>> 
>>>>> 
>>>>> https://github.com/apache/arrow/blob/master/python/manylinux1/scripts/build_boost.sh#L38
>>>>> <
>>>>> https://github.com/apache/arrow/blob/master/python/manylinux1/scripts/build_boost.sh#L38
>>>>>> 
>>>>> 
>>>>> After making change, the manylinux1 build still fails with the same error
>>>>> :(.
>>>>> 
>>>>> https://travis-ci.org/apache/arrow/jobs/498847622
>>>>> 
>>>>> Looks like the CI run downloads a prebuilt docker image. Do I need to
>>>>> update the docker image ? If yes, can you please point out the instructions
>>>>> for this ?
>>>>> 
>>>>> Thanks & regards,
>>>>> Ravindra.
>>>> 
>> 


Mime
View raw message