mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Olivier <cjolivie...@gmail.com>
Subject Re: [DISCUSS] Build OSX builds in CI (possibly with TravisCI).
Date Tue, 12 Dec 2017 17:10:38 GMT
googling seems to be full of running OSX (and even open-sourced PureDarwin)
in VMs. One could conceivably run a VM on an EC2 instance, right?

On Tue, Dec 12, 2017 at 9:01 AM kellen sunderland <
kellen.sunderland@gmail.com> wrote:

> It would be ideal if we could cover OSX in Jenkins, but the only solution
> that I'm aware of would require physical machines to be the workers.  I
> would be weakly opposed to having physical servers running on PRs.  The
> downsides that I see in order of importance:
>
> -  We can't autoscale physical hardware.   If we find that the load is too
> high we have to buy more machines.
> -  Security would be tricky, as they'd have to be connected to the internet
> and then to our Jekins master instance.  Connecting via a wired network
> would probably not be possible on most corporate networks as these machines
> are by definition running arbitrary code from the internet.  Many corporate
> sites have public wifi that this machine could potentially connect to, but
> then our PRs start failing if the wifi disconnects temporarily.  To connect
> to the master we would need to setup a vpn solution with endpoints in our
> vpc on AWS.  This is possible but would probably require a lot of security
> work.
> -  We can't just create a simple startup script or yaml file that is
> checked into GitHub to manage the machine.  Someone will actually have to
> physically administer the machine, apply updates, etc. which will make
> community ownership difficult.
>
> Specific to an OSX build:
> -  We can't virtualize OSX which means we'd only be able to cover one OSX
> build environment per physical device.  We couldn't target a matrix of OSX
> and Xcode versions as in Travis.
>
> -Kellen
>
> On Tue, Dec 12, 2017 at 5:46 PM, Chris Olivier <cjolivier01@gmail.com>
> wrote:
>
> > So why Travis when we could possibly use Jenkins?
> >
> > On Tue, Dec 12, 2017 at 7:59 AM Marco de Abreu <
> > marco.g.abreu@googlemail.com>
> > wrote:
> >
> > > Yes that's correct, Chris.
> > >
> > > Am 12.12.2017 4:46 nachm. schrieb "Chris Olivier" <
> cjolivier01@gmail.com
> > >:
> > >
> > > > A quick google search seems to indicate that Mac can be used as a
> > Jenkins
> > > > slave. Is this correct?
> > > >
> > > > On Tue, Dec 12, 2017 at 7:42 AM Steffen Rochel <
> > steffenrochel@gmail.com>
> > > > wrote:
> > > >
> > > > > +1 for #1 and #2
> > > > >
> > > > > I’m working on getting a MacPro to add to CI system.
> > > > > On Tue, Dec 12, 2017 at 1:43 AM kellen sunderland <
> > > > > kellen.sunderland@gmail.com> wrote:
> > > > >
> > > > > > Background:  TravisCI is a startup providing managed continuous
> > > > > > integration services with GitHub integration and YAML based
> > > > > configuration.
> > > > > > TravisCI is one of the few CI providers that will build a variety
> > of
> > > > > > OSX/MacOS builds for software projects.  Their pricing ranges
> from
> > > Free
> > > > > > (for open source, 1 concurrent job, to $489 monthly for 10
> > concurrent
> > > > > jobs).
> > > > > >
> > > > > > Problem: We’ve had a few OSX build issues slip into MXNet
master
> in
> > > the
> > > > > > past few weeks.  We’ve previously had a Travis CI based testing
> > > system
> > > > > that
> > > > > > would have caught these issues.
> > > > > >
> > > > > > Proposals so far:
> > > > > >
> > > > > > 1) Use TravisCI in it’s free mode for a very minimal sanity
check
> > on
> > > > OSX.
> > > > > > If we compile the program, and for example run C++ unit tests
> we’re
> > > > > > unlikely to run into problems with queued builds.  The total
> build
> > > time
> > > > > > here should be less than 15 minutes.  Configuration should be
> quite
> > > > > simple
> > > > > > and easy to maintain.  Error messages should also be obvious
to
> > > > > > contributors.
> > > > > > 2) Run clang in Linux with our current CI.  Building with clang
> > > should
> > > > > > take less than 10 minutes, should flush out a large subset of
the
> > > > issues
> > > > > > we’ve seen with OSX, and be quite easy to maintain.
> > > > > > 3) Run full test-suites in TravisCI, equaling the level of
> coverage
> > > we
> > > > > > provide to Linux in Jenkins.  This could require us to subscribe
> > to a
> > > > > > monthly package with Travis to ensure our build queue doesn’t
> grow
> > to
> > > > an
> > > > > > unacceptable length.  It may also require a volunteer to setup
> and
> > > > > maintain
> > > > > > long-term.
> > > > > >
> > > > > > I’d +1 #1 and #2 as I think those should be low-cost,
> low-maintence
> > > > > > solutions that should catch the majority of the problems we’ve
> seen
> > > > thus
> > > > > > far.
> > > > > >
> > > > > > -Kellen
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message