yetus-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From a.@apache.org
Subject [yetus] branch master updated: YETUS-683. Publish apache/yetus docker images on hub.docker.com
Date Thu, 03 Jan 2019 07:48:24 GMT
This is an automated email from the ASF dual-hosted git repository.

aw pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/yetus.git


The following commit(s) were added to refs/heads/master by this push:
     new 8c3c818  YETUS-683. Publish apache/yetus docker images on hub.docker.com
8c3c818 is described below

commit 8c3c818285ee4e03bf31c56f678924ca37cfaf6e
Author: Allen Wittenauer <aw@apache.org>
AuthorDate: Sun Nov 11 22:26:26 2018 -0800

    YETUS-683. Publish apache/yetus docker images on hub.docker.com
    
    Signed-off-by: Allen Wittenauer <aw@apache.org>
---
 .../documentation/in-progress/precommit-basic.md   |  6 +-
 .../documentation/in-progress/precommit-robots.md  | 59 ++++++++++++++-
 asf-site-src/source/downloads.html.md.erb          |  2 +-
 asf-site-src/source/yetus-docker-image.md          | 86 ++++++++++++++++++++++
 4 files changed, 147 insertions(+), 6 deletions(-)

diff --git a/asf-site-src/source/documentation/in-progress/precommit-basic.md b/asf-site-src/source/documentation/in-progress/precommit-basic.md
index 5e24d28..63d25f2 100644
--- a/asf-site-src/source/documentation/in-progress/precommit-basic.md
+++ b/asf-site-src/source/documentation/in-progress/precommit-basic.md
@@ -84,7 +84,7 @@ Build Tools:
 Automation and Isolation:
 
 * [Circle CI](https://www.circleci.com)
-* [Docker](https://www.docker.com) version 1.6.0+
+* [Docker](https://www.docker.com) version 1.7.0+
 * [Gitlab CI](https://www.gitlab.com)
 * [Jenkins](https://www.jenkins-ci.org)
 * [Travis CI](https://www.travis-ci.com)
@@ -329,10 +329,10 @@ NOTE: JAVA\_HOME is always appended to the list of JDKs in MultiJDK
mode.  If JA
 `test-patch` also has a mode to utilize Docker:
 
 ```bash
-$ test-patch. (other options) --docker
+$ test-patch (other options) --docker
 ```
 
-This will do some preliminary setup and then re-execute itself inside a Docker container.
 For more information on how to provide a custom Dockerfile and other Docker-specific features,
see [precommit-docker](../precommit-docker) for more information.
+This command will do some preliminary setup and then re-execute itself inside a Docker container.
 For more information on how to provide a custom Dockerfile and other Docker-specific features,
see the specific [precommit Docker support](../precommit-docker) page and the [Apache Yetus
Docker Hub Images](/yetus-docker-image) page for more information on the convenience Docker
images.
 
 # In Closing
 
diff --git a/asf-site-src/source/documentation/in-progress/precommit-robots.md b/asf-site-src/source/documentation/in-progress/precommit-robots.md
index b0fab18..faa487c 100644
--- a/asf-site-src/source/documentation/in-progress/precommit-robots.md
+++ b/asf-site-src/source/documentation/in-progress/precommit-robots.md
@@ -38,6 +38,23 @@ TRIGGER: ${CIRCLECI}=true
 
 Circle CI support in `test-patch` is limited to github.com.  Artifacts (the `--patch-dir`
directory) location needs to be handled set on the command line.  Linking to the logs is not
currently supported.
 
+To use the pre-built Apache Yetus Docker image from docker hub as the build environment,
use the following snippet in the `.circleci/config.yaml` file, substituting the tag for the
version of Apache Yetus that should be used and replacing the JAVA_HOME with the appropriate
version as bundled mentioned in the Dockerfile:
+
+```yaml
+jobs:
+  build:
+    docker:
+      - image: apache/yetus:0.9.0
+
+    environment:
+      JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+
+  ...
+```
+
+See also
+  * See also the source tree's `.circleci/config.yaml` for some tips and tricks.
+
 Gitlab CI
 =========
 
@@ -54,6 +71,21 @@ Artifacts, patch logs, etc are configured to go to a yetus-out directory
in the
 
 ```
 
+To use the pre-built Apache Yetus Docker image from docker hub as the build environment,
use the following snippet in the `.gitlab-ci.yml` file, substituting the tag for the version
of Apache Yetus that should be used and replacing the JAVA_HOME with the appropriate version
as bundled mentioned in the Dockerfile:
+
+```yaml
+job:
+  image: apache/yetus:0.9.0
+  allow_failure: true
+  variables:
+    JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+
+  ...
+```
+
+See also
+  * See also the source tree's `.gitlab-ci.yml` for some tips and tricks.
+
 Jenkins
 =======
 
@@ -66,9 +98,29 @@ There is some support for a few well known environment variables:
   * `${GIT_URL}` will trigger the same extra handling if 'github' or 'gitlab' appear in the
string.
   * If `${ghprbPullId}` is set, then test-patch will configure itself for a Github-style
PR.
 
+To use the pre-built Apache Yetus Docker image from docker hub as the build environment,
use the following snippet in the `Jenkinsfile`, substituting the tag for the version of Apache
Yetus that should be used and replacing the JAVA_HOME with the appropriate version as bundled
mentioned in the Dockerfile:
+
+```groovy
+pipeline {
+  agent {
+    docker {
+      image 'apache/yetus:0.9.0'
+      args '-v /var/run/docker.sock:/var/run/docker.sock'
+    }
+  }
+
+  environment {
+    JAVA_HOME = '/usr/lib/jvm/java-8-openjdk-amd64'
+  }
+
+  ...
+
+}
+
+```
 
 See also
-  * See also the source tree's Jenkinsfile for some tips and tricks.
+  * See also the source tree's `Jenkinsfile` for some tips and tricks.
   * [precommit-admin](precommit-admin), for special utilities built for Jenkins.
   * [GitHub Branch Source Plugin](https://wiki.jenkins.io/display/JENKINS/GitHub+Branch+Source+Plugin)
   * [GitHub Pull Request Builder Plugin](https://wiki.jenkins.io/display/JENKINS/GitHub+pull+request+builder+plugin)
@@ -83,9 +135,12 @@ Travis CI support will update the local checked out source repository
to include
 
 If `${ARTIFACTS_PATH}` is configured, then `--patch-dir` is set to the first listed directory
path.  However, links to the location logs must still be configured manually.
 
-Personalities will override the auto-detected Github repository information.  It may be necessary
to manually configure it in your .travis.yml file.
+Personalities will override the auto-detected Github repository information.  It may be necessary
to manually configure it in your `.travis.yml` file.
 
+As of this writing, it is not possible to make the Travis CI build environment use the Apache
Yetus pre-built docker images without using ` docker run` in the before_install phase.  Therefore,
using the image is the same as described in the [Apache Yetus Docker Hub Images](/yetus-docker-image)
page.
 
+See also
+  * See also the source tree's `.travis.yml` for some tips and tricks.
 
 Manual Configuration
 ====================
diff --git a/asf-site-src/source/downloads.html.md.erb b/asf-site-src/source/downloads.html.md.erb
index e3c0f11..27c362e 100644
--- a/asf-site-src/source/downloads.html.md.erb
+++ b/asf-site-src/source/downloads.html.md.erb
@@ -31,7 +31,7 @@ should verify the integrity of these artifacts via [the ASF procedure](https://w
 using the [KEYS file for our project](https://dist.apache.org/repos/dist/release/yetus/KEYS).
 
 In addition to the artifacts below, we publish convenience artifacts in some
-third-party repositories (e.g. Maven Central).
+third-party repositories (e.g. Maven Central) and [Docker Hub](../yetus-docker-image).
 
 ## Releases
 
diff --git a/asf-site-src/source/yetus-docker-image.md b/asf-site-src/source/yetus-docker-image.md
new file mode 100644
index 0000000..963cbf5
--- /dev/null
+++ b/asf-site-src/source/yetus-docker-image.md
@@ -0,0 +1,86 @@
+<!---
+  Licensed to the Apache Software Foundation (ASF) under one
+  or more contributor license agreements.  See the NOTICE file
+  distributed with this work for additional information
+  regarding copyright ownership.  The ASF licenses this file
+  to you under the Apache License, Version 2.0 (the
+  "License"); you may not use this file except in compliance
+  with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing,
+  software distributed under the License is distributed on an
+  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+  KIND, either express or implied.  See the License for the
+  specific language governing permissions and limitations
+  under the License.
+-->
+
+Convenience Docker Hub Images
+===========================
+
+While not official releases, the Apache Yetus project makes available two types of images
on hub.docker.com:
+
+  * apache/yetus-base
+
+    This image is the same as the 'built-in' Dockerfile when `--docker` is used without `--dockerfile`
 on the precommit command line.  It includes all of the pre-requisites as needed by the various
Apache Yetus components. It is located in `precommit/src/main/shell/test-patch-docker/`.
+
+  * apache/yetus
+
+    This image is the same as apache/yetus-base but includes a pre-built version of Apache
Yetus as part of the base OS image. In other words, qbt, releasedocmaker, shelldocs, test-patch,
etc., are in /usr/bin and available in the default path. It is generated from the Dockerfile
located in the root of the source and is built with the options provided in the hooks directory.
+
+Both images should be suitable to be used as a building block or even directly if your build
environment needs no other dependencies.  These images are especially useful for various CI
systems that require a Docker image to be used.
+
+Images are tagged such that 'master' represents the last successful Docker image build of
the master branch.  Images based off of the official source releases are tagged with a matching
version number (e.g., 0.9.0).  There is no 'latest' tagged image.  It is recommended that
users choose a stable tag so as not to be surprised by incompatible changes.
+
+File Access
+========
+
+All of the executables that Apache Yetus provides requires access to one or more directories.
 These directories should be provided on the command line via the volume flags to docker run.
For example, to run shelldocs against test-patch.sh:
+
+```bash
+docker run \
+  --rm \
+  -v /tmp/out:/output \
+  -v /src/precommit/src/main/shell:/input:ro \
+  apache/yetus:0.9.0 \
+    shelldocs --output /output/test-patch.md --input /input/test-patch.sh
+```
+
+In this example, we mount two volumes: one for input (marked read-only), one for output.
 After execution, shelldocs has processed test-patch.sh and generated a test-patch.md file
in /tmp/out on our local machine.
+
+Precommit also works, assuming that Apache Yetus image has all of your project's dependencies.
 For example:
+
+```bash
+docker run \
+  --rm \
+  -v /tmp/patchdir:/patchdir \
+  -v /src:/basedir \
+  apache/yetus:0.9.0 \
+  qbt \
+    --plugins=all \
+    --basedir=/basedir \
+    --patch-dir=/patchdir \
+    --project=yetus
+```
+If your project needs additional dependencies, it is trivial to build off of the Apache Yetus
image:
+
+```Dockerfile
+FROM apache/yetus:0.9.0
+RUN apt-get -q update && apt-get -q install -y \
+      clang \
+      libssl-dev \
+      valgrind \
+      zlib1g-dev
+```
+ ```bash
+docker build -t project/build:0.9.0 -f .
+```
+
+This example builds a docker image based off of Apache Yetus 0.9.0 but with the additions
of clang, some development libraries, and valgrind.  Now project/build:0.9.0 can be used instead
of apache/yetus:0.9.0 since it has all of Apache Yetus and the additions needed by our project.
+
+A Note About Precommit
+================
+
+test-patch and friends have direct support for Docker outside of the convenience images.
 That information is covered in-depth in the[precommit-docker](../precommit-docker) section.


Mime
View raw message