arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Krisztián Szűcs <szucs.kriszt...@gmail.com>
Subject Re: Travis CI delays
Date Sun, 29 Sep 2019 00:34:02 GMT
I'd like to mention that we have the option to reuse the buildbot
builders as well. I've put up a PR [1] which runs the exact same
builders (without the CUDA ones) what we have in the buildbot
setup [2]. The required GitHub Actions configuration is fairly
small [3].

[1]: https://github.com/apache/arrow/pull/5536
[2]: https://ci.ursalabs.org/#/builders
[3]: https://github.com/apache/arrow/pull/5536/files

On Sat, Sep 28, 2019 at 1:36 AM Francois Saint-Jacques <
fsaintjacques@gmail.com> wrote:

> Hello,
>
> I suggest we tackle https://jira.apache.org/jira/browse/ARROW-5801.
> For Rust, that would be
> https://jira.apache.org/jira/browse/ARROW-5809. Once ported to
> docker/docker-compose, it's trivial to activate github action for the
> same test (see https://github.com/apache/arrow/pull/5530). As I'm
> writing this email (4 hours after the PR was pushed), the travis
> checks are still not completed due to the queue. While the github
> action was completed after 22 minutes
> (https://github.com/apache/arrow/pull/5530/checks?check_run_id=239290338).
>
> François
>
>
> On Fri, Sep 27, 2019 at 11:22 AM Krisztián Szűcs
> <szucs.krisztian@gmail.com> wrote:
> >
> > On Fri, Sep 27, 2019 at 5:07 PM Andy Grove <andygrove73@gmail.com>
> wrote:
> >
> > > Thanks Krisztian. That's very helpful. I will create a CI page on the
> wiki
> > > and add this info.
> > >
> > > Does anyone have any objections to me trying out GitHub Actions for
> running
> > > the Rust tests on PR builds? I could try this out on my own fork first.
> > >
> > I think GitHub Actions is a good idea, especially for easier build setups
> > like Rust
> > has. Go, Node as similarly straightforward, so we could decommission the
> > travis
> > counterparts hopefully speeding up the rest of the builds a bit.
> >
> > >
> > > On Fri, Sep 27, 2019 at 7:40 AM Krisztián Szűcs <
> szucs.krisztian@gmail.com
> > > >
> > > wrote:
> > >
> > > > On Fri, Sep 27, 2019 at 3:25 PM Andy Grove <andygrove73@gmail.com>
> > > wrote:
> > > >
> > > > > I've been poking around on the Arrow website and wiki and I can't
> find
> > > > > documentation relating to CI. Do we have any documentation on how
> > > things
> > > > > work today or what the goals are?
> > > >
> > > > I don't think so.
> > > >
> > > > > For Rust builds it isn't immediately
> > > > > obvious why they are building on both Travis CI and Ursabot.
> > > > >
> > > > We have another thread [1], where we discuss multiple things about
> > > > Buildbot (ursabot). I've enabled the buildbot builder for rust
> because it
> > > > provides much quicker feedback than travis does.
> > > >
> > > > [1]:
> > > >
> > > >
> > >
> https://lists.apache.org/thread.html/02f7981176b67a12618b96b7d3b13e38b8f862e14c735dcf0ae359e0@%3Cdev.arrow.apache.org%3E
> > > >
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Sep 27, 2019 at 6:33 AM Wes McKinney <wesmckinn@gmail.com>
> > > > wrote:
> > > > >
> > > > > > FYI, using the kibble.dev link from INFRA-18533, it seems that
> this
> > > > > > September we're using about 15% of the ASF's total Travis CI
> capacity
> > > > > > (60 concurrent workers I think)
> > > > > >
> > > > > > https://imgur.com/a/oOrbPsj
> > > > > >
> > > > > > The highest is Apache Druid (incubating) at 18%, so we are #2.
> > > Suffice
> > > > > > to say the ASF's Travis couldn't accommodate us if we had twice
> as
> > > > > > many pull requests
> > > > > >
> > > > > > On Fri, Sep 27, 2019 at 7:26 AM Wes McKinney <
> wesmckinn@gmail.com>
> > > > > wrote:
> > > > > > >
> > > > > > > I've been sounding the alarm bells about this for a while.
We
> need
> > > to
> > > > > > > work to get ourselves off of Travis CI, but it is not going
to
> be
> > > > > > > easy.t
> > > >
> > > > > >
> > > > > > > On Thu, Sep 26, 2019 at 11:42 PM Micah Kornfield <
> > > > > emkornfield@gmail.com>
> > > > > > wrote:
> > > > > > > >
> > > > > > > > My understanding is the Travis CI queue is shared
among all
> > > apache
> > > > > > > > projects, and there are few including Arrow that make
heavy
> use
> > > of
> > > > > the
> > > > > > > > resources.  Hence, a lot of time waiting for jobs
to start.
> I
> > > > think
> > > > > > there
> > > > > > > > are some open JIRAs to finish dockerization of builds,
I
> don't
> > > know
> > > > > the
> > > > > > > > current status of finding alternative CI sources though.
> > > > > > > >
> > > > > > > > On Thu, Sep 26, 2019 at 10:24 PM Andy Grove <
> > > andygrove73@gmail.com
> > > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > I know this has been discussed in the past, and
I
> apologize for
> > > > not
> > > > > > paying
> > > > > > > > > attention at the time (and searching for arrow
+ travis in
> > > email
> > > > > > isn't very
> > > > > > > > > effective) but why does it take so long for our
Travis CI
> > > builds
> > > > > and
> > > > > > are
> > > > > > > > > there open JIRA issues related to this?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > >
> > > > > > > > > Andy.
> > > > > > > > >
> > > > > >
> > > > >
> > > >
> > >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message