hawq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roman Shaposhnik <ro...@shaposhnik.org>
Subject Re: Publishing HAWQ dev docker images
Date Wed, 28 Dec 2016 19:43:52 GMT
Hi!

On Tue, Dec 27, 2016 at 9:55 PM, Richard Guo <guofenglinux@gmail.com> wrote:
> Hi everyone,
>
> HAWQ team is building HAWQ dev docker images. The purpose is to provide an
> out-of-box way for developers to setup build and test environment for HAWQ.

First of all I think it is a great idea and will go a long way of
simplifying the developer
experience.

Based on my observations of how Docker containers are used for build
and development
purposes by various other projects at ASF I'd like to share some of my
experience:
   * while it may seem counterintuitive, Dockerfiles (for various
supported platforms) are
     much more important than a docker container on Docker Hub. This
is partially because
     despite all the deafening hype not everybody is in love with
Docker as much as
     the tech press would make you believe. Having a Dockerfile lets
folks who are not
     using Docker recreate a build environment based on what's in
there as part of RUN, etc.
     Which brings me to my next point:

   * extract your actual environment setup logic into a script that
can easily be run outside
     of Docker environment, then copy this script into a Docker
container and RUN it. See
     this example from Docker best practices:
          https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/#/add-or-copy

   * make sure to use your containers for actual builds on
builds.apache.org otherwise they
     tend to bitrot.

> A demo could be found on github hawq-docker
> <https://github.com/guofengrichard/hawq-docker> . Currently only CentOS 7
> is supported. CentOS 6 will be supported soon. It is based on Zhanwei's
> work.

This is a good start that needs to be folded into the HAWQ code base itself
and ideally hooked to the top level Make logic.

Also, if you could consider separating the logic into a separate script that'd
be super awesome and would allow us for a much better integration with
downstream consumers such as Linux distributions and Bigtop.

> The idea is to predefine all the environment setup steps in the dockerfile
> and then build the image from dockerfile with tools provided by docker.
> After that, users can simply create containers with the docker image and
> then do the HAWQ build and test jobs.
> Also a Makefile is provided to simplify this process. Please refer to the
> README in the github repository for more details.
>
> Regarding the place to host the docker images and dockerfiles, does anyone
> have any idea? Comments and discussions are welcomed.

These types of build environments are full of GPL things and thus need to be
kept separately from ASF artifacts. I suggest the same approach we used in
Bigtop: a community owned, public GitHub repo. E.g.:
     https://hub.docker.com/u/bigtop/

Initially I'd suggest a pretty tight ownership of who can modify
containers in your
account. The best way we found so far is to have a single volunteer
who establishes
credentials and makes sure to push containers into the account everytime there's
a change. If you couple it with setting the email on the account to
private@hawq.i.a.o
you've got yourself a pretty secure AND community friendly setup.

In fact, while checking if hawq was available on Docker hub I took the
liberty of creating
an account for you ;-) Let me know if you like it:
     https://hub.docker.com/u/hawq/

Thanks,
Roman.

Mime
View raw message