bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Tzolov <christian.tzo...@gmail.com>
Subject Re: Bigtop Docker image with Dockerfile.
Date Wed, 25 Feb 2015 17:57:02 GMT
Hi Konstantin,

Below are some of my findings. Note that so far i've played only with
CentOS6.

- Missing python-setuptools and asciidoc packages. Don't remember which
project needed them.
- To build spark-rpm you need install scala and set the SCALA_HOME env.
- Due to https://issues.apache.org/jira/browse/CRUNCH-500 you can't use
Maven > 3.1. I've fixed this in the Crunch project but will not be
available until Crunch 0.12 release. For the time being can create a patch
for Crunch's do-component-build? In the Dockerfile I've downgraded maven to
3.0.5
- Hue (hue-ui) requires non root user to build the rpm.

----

I've tried
https://cwiki.apache.org/confluence/display/BIGTOP/How+to+build+Bigtop using
the centos image. It seems like the /etc/profile.d/bigtop.sh is missing?
The 'toolchain' requires java to be installed first (and JAVA_HOME to be
set). This in turn requires 'sudo' package. For Spark you need to install
Scala and set SCALA_HOME. And for Hue you need to create sudo, passwordless
user...
Maybe with the ubunto image is better (haven't tried it) but for me the
above requires too many steps to prepare the environment.

Hope this helps.

Cheers,
Christian





On Wed, Feb 25, 2015 at 4:12 AM, Konstantin Boudnik <cos@apache.org> wrote:

> Hi Christian.
>
> On Tue, Feb 24, 2015 at 09:24AM, Christian Tzolov wrote:
> > Hi Konstantin,
> >
> > I see your point. My goal however was to generate RPMs for some projects
> > (Spark particularly) and experiment with different versions and
> > configurations.
> >
> > I could not find a proper doc explaining how to bootstrap with Bigtop.
> The
> > wiki pages like (
> >
> https://cwiki.apache.org/confluence/display/BIGTOP/Building+Bigtop+on+RHEL7
> )
> > are outdated, missing libraries and stil refering to the 'make' tool.
>
> Yup, it is - I just recently figured this out and filed the JIRA
>    https://issues.apache.org/jira/browse/BIGTOP-1665
> which you're welcome to take, if you want.
>
> > I've tried the './gradlew toolchain' to build an environment but it still
> > requires some libs to be preinstalled so i've decided to keep everything
> in
> > a single Dockerfile.
>
> Could you please share what libs were missing? It'd be great to get it
> fixed.
> The issue here is the we are moving forward quite fast and sometimes devs
> env.
> isn't exactly pristine, so think like missing packages might go unnoticed.
>
> > Concerning only the packaging tasks I guess 2 environments (for RPM and
> > DEB) are enough?
>
> Yes, that's all Bigtop does on the packaging side.
>
> I have put together an initial draft for new build procedure
>   https://cwiki.apache.org/confluence/display/BIGTOP/How+to+build+Bigtop
>
> and you're welcome to update/add to it. Hope it helps.
>
>   Cos
>
> > On Mon, Feb 23, 2015 at 8:39 PM, Konstantin Boudnik <cos@apache.org>
> wrote:
> >
> > > Thanks Christian.
> > >
> > > We have considered this on a multiple occasions and as you found for
> > > yourself
> > > the Dockerfile file is pretty simple, so the consensus was to just
> > > check-in a
> > > Gradle code for its generation, hence making containers preparation as
> > > stateless as possible.
> > >
> > > Roman should have this patch somewhere, not sure if he's ready to
> commit
> > > yet.
> > > But we always can use your approach is a backup plan, although we'd
> need
> > > Dockerfiles for all supported Linuxes.
> > >
> > >   Cos
> > >
> > > On Mon, Feb 23, 2015 at 07:45PM, Christian Tzolov wrote:
> > > >    I've put together a Docker container, equipped with all necessary
> > > tools
> > > >    and libraries and ready to generate RPM packages. Here are the
> > > >    Dockerhub/Github references:
> > > >    https://registry.hub.docker.com/u/tzolov/bigtop-centos
> > > >    https://github.com/tzolov/bigtop-centos
> > > >    There are a couple of existing bigtop images in Dockerhub but
> neither
> > > of
> > > >    them provides neither description nor a Dockerfile.
> > > >
> > > >    For simplicity I've decided to keep it as as single image for now.
> > > >    Cheers,
> > > >    Christian
> > >
>
>

Mime
View raw message