beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From git-site-r...@apache.org
Subject [beam] branch asf-site updated: Publishing website 2019/09/17 19:06:59 at commit 6d56040
Date Tue, 17 Sep 2019 19:07:12 GMT
This is an automated email from the ASF dual-hosted git repository.

git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/asf-site by this push:
     new 497c18c  Publishing website 2019/09/17 19:06:59 at commit 6d56040
497c18c is described below

commit 497c18ce7f7aa06d5c45ff8c9499180a6bd17f88
Author: jenkins <builds@apache.org>
AuthorDate: Tue Sep 17 19:06:59 2019 +0000

    Publishing website 2019/09/17 19:06:59 at commit 6d56040
---
 .../contribute/release-guide/index.html            | 111 ++++++++++++++++++++-
 1 file changed, 110 insertions(+), 1 deletion(-)

diff --git a/website/generated-content/contribute/release-guide/index.html b/website/generated-content/contribute/release-guide/index.html
index 803566f..9385dad 100644
--- a/website/generated-content/contribute/release-guide/index.html
+++ b/website/generated-content/contribute/release-guide/index.html
@@ -271,6 +271,7 @@
     <ul>
       <li><a href="#deploy-artifacts-to-maven-central-repository">Deploy artifacts
to Maven Central Repository</a></li>
       <li><a href="#deploy-python-artifacts-to-pypi">Deploy Python artifacts
to PyPI</a></li>
+      <li><a href="#deploy-sdk-docker-images-to-dockerhub">Deploy SDK docker
images to DockerHub</a></li>
       <li><a href="#git-tag">Git tag</a></li>
       <li><a href="#merge-website-pull-request">Merge website pull request</a></li>
       <li><a href="#mark-the-version-as-released-in-jira">Mark the version as
released in JIRA</a></li>
@@ -327,6 +328,7 @@ limitations under the License.
           <li><a href="#submit-your-gpg-public-key-into-mit-pgp-public-key-server"
id="markdown-toc-submit-your-gpg-public-key-into-mit-pgp-public-key-server">Submit your
GPG public key into MIT PGP Public Key Server</a></li>
           <li><a href="#website-development-setup" id="markdown-toc-website-development-setup">Website
development setup</a></li>
           <li><a href="#register-to-pypi" id="markdown-toc-register-to-pypi">Register
to PyPI</a></li>
+          <li><a href="#login-to-dockerhub" id="markdown-toc-login-to-dockerhub">Login
to DockerHub</a></li>
         </ul>
       </li>
       <li><a href="#create-a-new-version-in-jira" id="markdown-toc-create-a-new-version-in-jira">Create
a new version in JIRA</a></li>
@@ -364,6 +366,7 @@ limitations under the License.
           <li><a href="#build-and-stage-java-artifacts-with-gradle" id="markdown-toc-build-and-stage-java-artifacts-with-gradle">Build
and stage Java artifacts with Gradle</a></li>
           <li><a href="#stage-source-release-on-distapacheorg" id="markdown-toc-stage-source-release-on-distapacheorg">Stage
source release on dist.apache.org</a></li>
           <li><a href="#stage-python-binaries-on-distapacheorg" id="markdown-toc-stage-python-binaries-on-distapacheorg">Stage
python binaries on dist.apache.org</a></li>
+          <li><a href="#stage-sdk-images-on-hubdockercom" id="markdown-toc-stage-sdk-images-on-hubdockercom">Stage
SDK images on hub.docker.com</a></li>
         </ul>
       </li>
       <li><a href="#build-and-stage-python-wheels" id="markdown-toc-build-and-stage-python-wheels">Build
and stage python wheels</a></li>
@@ -395,6 +398,7 @@ limitations under the License.
           <li><a href="#deploy-source-release-to-distapacheorg" id="markdown-toc-deploy-source-release-to-distapacheorg">Deploy
source release to dist.apache.org</a></li>
         </ul>
       </li>
+      <li><a href="#deploy-sdk-docker-images-to-dockerhub" id="markdown-toc-deploy-sdk-docker-images-to-dockerhub">Deploy
SDK docker images to DockerHub</a></li>
       <li><a href="#git-tag" id="markdown-toc-git-tag">Git tag</a></li>
       <li><a href="#merge-website-pull-request" id="markdown-toc-merge-website-pull-request">Merge
website pull request</a></li>
       <li><a href="#mark-the-version-as-released-in-jira" id="markdown-toc-mark-the-version-as-released-in-jira">Mark
the version as released in JIRA</a></li>
@@ -472,6 +476,7 @@ please follow <a href="https://help.github.com/articles/creating-a-personal-acce
   <li>GPG pass phrase (see the next section);</li>
   <li>Apache ID and Password;</li>
   <li>GitHub ID and Password.</li>
+  <li>DockerHub ID and Password. (You should be a member of maintainer team; email
at dev@ if you are not.)</li>
 </ul>
 
 <h3 id="one-time-setup-instructions">One-time setup instructions</h3>
@@ -633,6 +638,32 @@ Automation Reliability</a>.</p>
 
 <p>Release manager needs to have an account with PyPI. If you need one, <a href="https://pypi.python.org/account/register/">register
at PyPI</a>. You also need to be a maintainer (or an owner) of the <a href="https://pypi.python.org/pypi/apache-beam">apache-beam</a>
package in order to push a new release. Ask on the mailing list for assistance.</p>
 
+<h4 id="login-to-dockerhub">Login to DockerHub</h4>
+<p>Run following command manually. It will ask you to input your DockerHub ID and password
if 
+authorization info cannot be found from ~/.docker/config.json file.</p>
+<div class="highlighter-rouge"><pre class="highlight"><code>docker login
docker.io
+</code></pre>
+</div>
+<p>After successful login, authorization info will be stored at ~/.docker/config.json
file. For example,</p>
+<div class="highlighter-rouge"><pre class="highlight"><code>"https://index.docker.io/v1/":
{
+   "auth": "aGFubmFoamlhbmc6cmtkdGpmZ2hrMTIxMw=="
+}
+</code></pre>
+</div>
+<p>Release managers should have push permission; please ask for help at dev@.</p>
+<div class="highlighter-rouge"><pre class="highlight"><code>From: Release
Manager
+To: dev@beam.apache.org
+Subject: DockerHub Push Permission
+
+Hi DockerHub Admins
+
+I need push permission to proceed with release, can you please add me to maintainer team?
+My docker hub ID is: xxx
+
+Thanks,
+Release Manager
+</code></pre>
+</div>
 <h3 id="create-a-new-version-in-jira">Create a new version in JIRA</h3>
 
 <p>When contributors resolve an issue in JIRA, they are tagging it with a release that
will contain their changes. With the release currently underway, new issues should be resolved
against a subsequent future release. Therefore, you should create a release item for this
subsequent release, as follows:</p>
@@ -685,7 +716,8 @@ Automation Reliability</a>.</p>
         <p>Change and commit version number in release branch:</p>
 
         <p><a href="https://github.com/apache/beam/blob/release-2.6.0/sdks/python/apache_beam/version.py#L21">version.py</a>,

-<a href="https://github.com/apache/beam/blob/release-2.6.0/runners/google-cloud-dataflow-java/build.gradle#L39">build.gradle</a></p>
+<a href="https://github.com/apache/beam/blob/release-2.6.0/runners/google-cloud-dataflow-java/build.gradle#L39">build.gradle</a>,

+<a href="https://github.com/apache/beam/blob/release-2.16.0/gradle.properties#L27">gradle.properties</a></p>
       </li>
     </ol>
   </li>
@@ -759,6 +791,7 @@ git commit -m "Moving to ${NEXT_VERSION_IN_BASE_BRANCH}-SNAPSHOT on master
branc
 
     <div class="highlighter-rouge"><pre class="highlight"><code>DEV=${RELEASE}.dev
 sed -i -e "s/${DEV}/${RELEASE}/g" sdks/python/apache_beam/version.py
+sed -i -e "s/${DEV}/${RELEASE}/g" gradle.properties
 sed -i -e "s/'beam-master-.*'/'beam-${RELEASE}'/g" runners/google-cloud-dataflow-java/build.gradle
 </code></pre>
     </div>
@@ -1063,6 +1096,7 @@ When prompted for a description, enter “Apache Beam, version X, release
candid
       </li>
       <li>Stage source release into dist.apache.org dev <a href="https://dist.apache.org/repos/dist/dev/beam/">repo</a>.</li>
       <li>Stage,sign and hash python binaries into dist.apache.ord dev repo python
dir</li>
+      <li>Stage SDK docker images to <a href="https://hub.docker.com/u/apachebeam">https://hub.docker.com/u/apachebeam</a>.</li>
       <li>Create a PR to update beam and beam-site, changes includes:
         <ul>
           <li>Copy python doc into beam-site</li>
@@ -1195,6 +1229,53 @@ svn commit
 
 <p>Verify that files are <a href="https://dist.apache.org/repos/dist/dev/beam">present</a>.</p>
 
+<h4 id="stage-sdk-images-on-hubdockercom">Stage SDK images on hub.docker.com</h4>
+<ul>
+  <li>Build Python images and push to DockerHub.</li>
+</ul>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>./gradlew :sdks:python:container:buildAll
-Pdocker-tag=${RELEASE}_rc{RC_NUM}
+
+PYTHON_VER=("python2.7" "python3.5" "python3.6" "python3.7")
+for ver in "${PYTHON_VER[@]}"; do
+   docker push apachebeam/${ver}_sdk:${RELEASE}_rc{RC_NUM} &amp;
+done
+</code></pre>
+</div>
+
+<ul>
+  <li>Build Java images and push to DockerHub.</li>
+</ul>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>./gradlew :sdks:java:container:dockerPush
-Pdocker-tag=${RELEASE}_rc{RC_NUM}
+</code></pre>
+</div>
+
+<ul>
+  <li>Build Go images and push to DockerHub.</li>
+</ul>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>./gradlew :sdks:go:container:dockerPush
-Pdocker-tag=${RELEASE}_rc{RC_NUM}
+</code></pre>
+</div>
+
+<p>Clean up images from local</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>for ver in
"${PYTHON_VER[@]}"; do
+   docker rmi -f apachebeam/${ver}_sdk:${RELEASE}_rc{RC_NUM}
+done
+docker rmi -f apachebeam/java_sdk:${RELEASE}_rc{RC_NUM}
+docker rmi -f apachebeam/go_sdk:${RELEASE}_rc{RC_NUM}
+</code></pre>
+</div>
+
+<p>How to find images:</p>
+<ol>
+  <li>Visit <a href="https://hub.docker.com/u/apachebeam">https://hub.docker.com/u/apachebeam</a></li>
+  <li>Visit each repository and navigate to <em>tags</em> tab.</li>
+  <li>Verify images are pushed with tags: ${RELEASE}_rc{RC_NUM}</li>
+</ol>
+
 <h3 id="build-and-stage-python-wheels">Build and stage python wheels</h3>
 
 <p>There is a wrapper repo <a href="https://github.com/apache/beam-wheels">beam-wheels</a>
to help build python wheels.</p>
@@ -1358,6 +1439,7 @@ all major features and bug fixes, and all known issues.</p>
   <li>Maven artifacts deployed to the staging repository of <a href="https://repository.apache.org/content/repositories/">repository.apache.org</a></li>
   <li>Source distribution deployed to the dev repository of <a href="https://dist.apache.org/repos/dist/dev/beam/">dist.apache.org</a></li>
   <li>Website pull request proposed to list the <a href="/get-started/downloads/">release</a>,
publish the <a href="https://beam.apache.org/releases/javadoc/">Java API reference manual</a>,
and publish the <a href="https://beam.apache.org/releases/pydoc/">Python API reference
manual</a>.</li>
+  <li>Docker images are published to <a href="https://hub.docker.com/u/apachebeam">DockerHub</a>
with tags: {RELEASE}_rc{RC_NUM}.</li>
 </ol>
 
 <p>You can (optionally) also do additional verification by:</p>
@@ -1367,6 +1449,12 @@ all major features and bug fixes, and all known issues.</p>
   <li>Check signatures (e.g. <code class="highlighter-rouge">gpg --verify apache-beam-1.2.3-python.zip.asc
apache-beam-1.2.3-python.zip</code>)</li>
   <li><code class="highlighter-rouge">grep</code> for legal headers in
each file.</li>
   <li>Run all jenkins suites and include links to passing tests in the voting email.
(Select “Run with parameters”)</li>
+  <li>Pull docker images to make sure they are pullable.
+    <div class="highlighter-rouge"><pre class="highlight"><code>docker
pull {image_name}
+docker pull apachebeam/python3.5_sdk:2.16.0_rc1
+</code></pre>
+    </div>
+  </li>
 </ol>
 
 <hr />
@@ -1396,6 +1484,7 @@ The complete staging area is available for your review, which includes:
 * Java artifacts were built with Maven MAVEN_VERSION and OpenJDK/Oracle JDK JDK_VERSION.
 * Python artifacts are deployed along with the source release to the dist.apache.org [2].
 * Validation sheet with a tab for 1.2.3 release to help with validation [9].
+* Docker images puhlished to Docker Hub [10].
 
 The vote will be open for at least 72 hours. It is adopted by majority approval, with at
least 3 PMC affirmative votes.
 
@@ -1411,6 +1500,7 @@ Release Manager
 [7] https://github.com/apache/beam-site/pull/...
 [8] https://github.com/apache/beam/pull/...
 [9] https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=...
+[10] https://hub.docker.com/u/apachebeam
 </code></pre>
 </div>
 
@@ -1789,6 +1879,25 @@ delete the <code class="highlighter-rouge">.asc</code>,
<code class="highlighter
 
 <p>Make sure the download address for last release version is upldaed, <a href="https://github.com/apache/beam-site/pull/478">example
PR</a>.</p>
 
+<h3 id="deploy-sdk-docker-images-to-dockerhub">Deploy SDK docker images to DockerHub</h3>
+<p>TODO(hannahjiang): change link to master branch after #9560 is merged.</p>
+
+<ul>
+  <li>Script: <a href="https://github.com/Hannah-Jiang/beam/blob/release_script_for_containers/release/src/main/scripts/publish_docker_images.sh">publish_docker_images.sh</a></li>
+  <li>Usage
+    <div class="highlighter-rouge"><pre class="highlight"><code>./beam/release/src/main/scripts/publish_docker_images.sh
+</code></pre>
+    </div>
+    <p>Verify that:</p>
+  </li>
+  <li>Images are published at <a href="https://hub.docker.com/u/apachebeam">DockerHub</a>
with tags {RELEASE} and <em>latest</em>.</li>
+  <li>Images with <em>latest</em> tag are pointing to current release by
confirming
+    <ol>
+      <li>Digest of the image with <em>latest</em> tag is the same as the
one with {RELEASE} tag.</li>
+    </ol>
+  </li>
+</ul>
+
 <h3 id="git-tag">Git tag</h3>
 
 <p>Create and push a new signed tag for the released version by copying the tag for
the final release candidate, as follows:</p>


Mime
View raw message