bigtop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Boudnik <>
Subject Re: Bigtop environment setup
Date Sun, 23 Sep 2012 07:06:35 GMT
On Fri, Sep 21, 2012 at 09:20PM, Bruno MahИ wrote:
> On 09/19/2012 06:32 PM, Roman Shaposhnik wrote:
> >On Wed, Sep 19, 2012 at 6:00 PM, Anatoli Fomenko <> wrote:
> >>I found that in order to avoid unnecessary build failures I need to quickly set
up additional
> >>VMs for Bigtop supported platforms. From my experience with Precise, I would
say that it's a task that may take time.
> >>
> >>Any suggestions how it could be accelerated?
> >
> >You're raising a very good point, actually. In fact I've run into
> >this very issue while trying to configure an extra Jenkins
> >slave for bigtop01.
> >
> >Now, in the ideal world, all the build time dependencies
> >that we have would be packaged and we'd express the
> >fact that we depend on them via the very same packages
> >that we're maintaining. That's what RPM's BuildRequires:
> >and DEB's Build-Depends: fields are for -- to tell you
> >explicitly what's required to be installed on the system
> >before you can do the build of the package.
> >
> >Then you'd use the tools like:
> >     apt-get build-dep
> >     yum-builddep
> >to satisfy all the dependencies and you're done.
> >
> >Now, this works great in the environment where
> >you already have source packages which you
> >can give to apt-get build-dep/yum-builddep
> >
> >But Bigtop has to be bootstrapped from the source.
> >We can't assume existence of source packages.
> >
> >So here's the question to the bigger Bigtop
> >community -- how do we want to proceed to
> >manage repeatable build environments for
> >our packages?
> >
> >The options I see are:
> >    #1 maintain a parallel (very shallow) collection
> >         of puppet code that would, essentially,
> >         manage our "build slaves"
> >    #2 do #1 but automate it in such a way that
> >         the info actually gets harvested from
> >         spec/conrol files
> >
> >Thoughts?
> >
> >Thanks,
> >Roman.
> >
> #1 is nice since it can deal with non-packaging issue. But it still
> require people to install and know how to deal with puppet. From a
> dev point of view we also need to remember to not use the latest
> features since some OS lag significantly in term of versions of
> puppet available.
> #2 is also nice since it can be dealt with the usual set of tools.
> But it still requires some effort on users. Also some dependencies
> are not and will probably never be available as packages (ex: Oracle
> JDK).
> I also don't think there is one and only one solution.
> My setup at home is quite different from the bigtop01 one.
> And once you are familiar enough with Apache Bigtop and know how to
> set it up, you may find options #1 and #2 probably not well adapted
> to your situation.
> So this leads me to think about option #3: VMs.
> Tools like Boxgrinder and Oz can deal with multiple OSes and can
> create local images as well as push them to the cloud.
> The build would be repeatable and would not require any effort from
> the end user (apart maybe providing Oracle JDK, but that would have
> to be the case whatever the solution). Future contributors would
> just need to boot their VM to get started and hopefully ease
> contribution.
> Thoughts?

(writing at the end of the last post feels totally unnatural, but for the
benefit of the future readers I will comply :)

I think boxed environments (like VMs) are an overkill. One of the issues here
(as Anatoli pointed out) is non-even support of all OSes. Say, BoxGrinder has
some issues with Ubuntu, etc.

I suppose a ol' proven toolchain type of environment that can automatically
bootstrap upon a fresh install (or update itself: think Maven model) and pull
whatever apps are required in whatever form they might exist. Say, JDK (or
Groovy or else) can be downloaded as a tarball, given that a more suitable
packaging isn't available, etc. Such approach would be more fluid than a
somewhat rigid VMs, that would have to be updated periodically, versioned,

Another benefit of a toolchain is that BigTop packages might have to
redistribute/wrap some of the tools for later use once a package is installed
on a customer's system.

So, in other words, #2 above (or some modification of it) looks more appealing to me.

  Take care,
Konstantin (Cos) Boudnik
2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622

Disclaimer: Opinions expressed in this email are those of the author, and do
not necessarily represent the views of any company the author might be
affiliated with at the moment of writing.

View raw message