arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wes McKinney <wesmck...@gmail.com>
Subject Re: Boost and manylinux CI builds
Date Sun, 03 Mar 2019 23:15:42 GMT
hi Ravindra,

Can we document this (just by copy-pasting what you wrote) on the wiki
or someplace for future work that may touch the manylinux package
builds? This might be a bit more discoverable than going through the
email logs

Thanks!
Wes

On Fri, Mar 1, 2019 at 9:58 PM Ravindra Pindikura <ravindra@dremio.com> wrote:
>
> Thanks Uwe.
>
> For the record (in case someone needs to do it again), these are the steps :
>
> 1. Make the change in build_boost.sh
>
> 2. Setup an account on quay.io <http://quay.io/> and link to your GitHub account
>
> 3. In quay.io <http://quay.io/>,  Add a new repository using :
>
> A. Link to GitHub repository push
> B. Trigger build on changes to a specific branch (eg. myquay) of the repo (eq. pravindra/arrow)
> C. Set Dockerfile location to "/python/manylinux1/Dockerfile-x86_64_base”
> D. Set Context location to "/python/manylinux1”
>
> 4. Push change (in step 1) to the branch specified in step 3B
>
> This should trigger a build in quay.io <http://quay.io/>, the build takes about
2 hrs to finish.
>
> 5. Add a tag “latest” to the build after step 4 finishes, save the URL of the build
(eg. quay.io/pravindra <http://quay.io/pravindra/arrow_manylinux1_x86_64_base:latest>/arrow_manylinux1_x86_64_base:latest
<http://quay.io/pravindra/arrow_manylinux1_x86_64_base:latest>)
>
> 6. In your arrow PR,
>
> - include the change from 1.
> - update travis_script_manylinux.sh to point to the location from step 5.
>
> Thanks & regards,
> Ravindra.
>
> > On Feb 27, 2019, at 3:02 PM, Uwe L. Korn <uwelk@xhochy.com> wrote:
> >
> > Hello Ravindra,
> >
> > simplest thing would be when you open a pull request and I can then pick this up
and push it to my personal fork. Then a new image is built on quay.io. Otherwise, you can
also activate quay.io on your fork to get the docker image to build.
> >
> > Uwe
> >
> > On Wed, Feb 27, 2019, at 8:41 AM, Krisztián Szűcs wrote:
> >> Hi Ravindra!
> >>
> >> You'll need to rebuild the docker image and change this line accordingly:
> >> https://github.com/apache/arrow/blob/master/ci/travis_script_manylinux.sh#L57
> >>
> >> On Wed, Feb 27, 2019 at 8:29 AM Ravindra Pindikura <ravindra@dremio.com>
> >> wrote:
> >>
> >>> Hi,
> >>>
> >>> I added an include for boost header file in gandiva. This compiles on
> >>> ubuntu/Mac/windows, but fails with the manylinux CI entry.
> >>>
> >>> I’m getting a compilation failure :
> >>>
> >>> https://travis-ci.org/apache/arrow/jobs/498718755 <
> >>> https://travis-ci.org/apache/arrow/jobs/498718755>
> >>> /arrow/cpp/src/gandiva/decimal_xlarge.cc:29:44: fatal error:
> >>> boost/multiprecision/cpp_int.hpp: No such file or directory
> >>> #include "boost/multiprecision/cpp_int.hpp"
> >>> ^
> >>> compilation terminated.
> >>>
> >>>
> >>>
> >>> @xhocy and @kszucs pointed out the manylinux1 image has a very minimal
> >>> boost, and doesn’t include the multi precision files. So, the script that
> >>> builds boost for manylinux1 needs to be updated for this.
> >>>
> >>>
> >>>
> >>> https://github.com/apache/arrow/blob/master/python/manylinux1/scripts/build_boost.sh#L38
> >>> <
> >>> https://github.com/apache/arrow/blob/master/python/manylinux1/scripts/build_boost.sh#L38
> >>>>
> >>>
> >>> After making change, the manylinux1 build still fails with the same error
> >>> :(.
> >>>
> >>> https://travis-ci.org/apache/arrow/jobs/498847622
> >>>
> >>> Looks like the CI run downloads a prebuilt docker image. Do I need to
> >>> update the docker image ? If yes, can you please point out the instructions
> >>> for this ?
> >>>
> >>> Thanks & regards,
> >>> Ravindra.
> >>
>

Mime
View raw message