bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Linnell <plinn...@apache.org>
Subject Re: Bigtop Docker image with Dockerfile.
Date Thu, 26 Feb 2015 23:07:02 GMT
On Wed, 25 Feb 2015 18:57:02 +0100
Christian Tzolov <christian.tzolov@gmail.com> wrote:

> Hi Konstantin,
> 
> Below are some of my findings. Note that so far i've played only with
> CentOS6.
> 
> - Missing python-setuptools and asciidoc packages. Don't remember
> which project needed them.
> - To build spark-rpm you need install scala and set the SCALA_HOME
> env.
> - Due to https://issues.apache.org/jira/browse/CRUNCH-500 you can't
> use Maven > 3.1. I've fixed this in the Crunch project but will not be
> available until Crunch 0.12 release. For the time being can create a
> patch for Crunch's do-component-build? In the Dockerfile I've
> downgraded maven to 3.0.5
> - Hue (hue-ui) requires non root user to build the rpm.

Hue wanting a non-root user to build is a feature not a bug :)  

I've learned the hard way in packaging: *Never* ever build packages as
root.


> 
> ----
> 
> I've tried
> https://cwiki.apache.org/confluence/display/BIGTOP/How+to+build+Bigtop
> using the centos image. It seems like the /etc/profile.d/bigtop.sh is
> missing? The 'toolchain' requires java to be installed first (and
> JAVA_HOME to be set). This in turn requires 'sudo' package. For Spark
> you need to install Scala and set SCALA_HOME. And for Hue you need to
> create sudo, passwordless user...
> Maybe with the ubunto image is better (haven't tried it) but for me
> the above requires too many steps to prepare the environment.
> 
> Hope this helps.
> 
> Cheers,
> Christian
> 
> 
> 
> 
> 
> On Wed, Feb 25, 2015 at 4:12 AM, Konstantin Boudnik <cos@apache.org>
> wrote:
> 
> > Hi Christian.
> >
> > On Tue, Feb 24, 2015 at 09:24AM, Christian Tzolov wrote:
> > > Hi Konstantin,
> > >
> > > I see your point. My goal however was to generate RPMs for some
> > > projects (Spark particularly) and experiment with different
> > > versions and configurations.
> > >
> > > I could not find a proper doc explaining how to bootstrap with
> > > Bigtop.
> > The
> > > wiki pages like (
> > >
> > https://cwiki.apache.org/confluence/display/BIGTOP/Building+Bigtop+on+RHEL7
> > )
> > > are outdated, missing libraries and stil refering to the 'make'
> > > tool.
> >
> > Yup, it is - I just recently figured this out and filed the JIRA
> >    https://issues.apache.org/jira/browse/BIGTOP-1665
> > which you're welcome to take, if you want.
> >
> > > I've tried the './gradlew toolchain' to build an environment but
> > > it still requires some libs to be preinstalled so i've decided to
> > > keep everything
> > in
> > > a single Dockerfile.
> >
> > Could you please share what libs were missing? It'd be great to get
> > it fixed.
> > The issue here is the we are moving forward quite fast and
> > sometimes devs env.
> > isn't exactly pristine, so think like missing packages might go
> > unnoticed.
> >
> > > Concerning only the packaging tasks I guess 2 environments (for
> > > RPM and DEB) are enough?
> >
> > Yes, that's all Bigtop does on the packaging side.
> >
> > I have put together an initial draft for new build procedure
> >   https://cwiki.apache.org/confluence/display/BIGTOP/How+to+build+Bigtop
> >
> > and you're welcome to update/add to it. Hope it helps.
> >
> >   Cos
> >
> > > On Mon, Feb 23, 2015 at 8:39 PM, Konstantin Boudnik
> > > <cos@apache.org>
> > wrote:
> > >
> > > > Thanks Christian.
> > > >
> > > > We have considered this on a multiple occasions and as you
> > > > found for yourself
> > > > the Dockerfile file is pretty simple, so the consensus was to
> > > > just check-in a
> > > > Gradle code for its generation, hence making containers
> > > > preparation as stateless as possible.
> > > >
> > > > Roman should have this patch somewhere, not sure if he's ready
> > > > to
> > commit
> > > > yet.
> > > > But we always can use your approach is a backup plan, although
> > > > we'd
> > need
> > > > Dockerfiles for all supported Linuxes.
> > > >
> > > >   Cos
> > > >
> > > > On Mon, Feb 23, 2015 at 07:45PM, Christian Tzolov wrote:
> > > > >    I've put together a Docker container, equipped with all
> > > > > necessary
> > > > tools
> > > > >    and libraries and ready to generate RPM packages. Here are
> > > > > the Dockerhub/Github references:
> > > > >    https://registry.hub.docker.com/u/tzolov/bigtop-centos
> > > > >    https://github.com/tzolov/bigtop-centos
> > > > >    There are a couple of existing bigtop images in Dockerhub
> > > > > but
> > neither
> > > > of
> > > > >    them provides neither description nor a Dockerfile.
> > > > >
> > > > >    For simplicity I've decided to keep it as as single image
> > > > > for now. Cheers,
> > > > >    Christian
> > > >
> >
> >

Hope that helps,
Peter 

Mime
View raw message