From commits-return-4227-archive-asf-public=cust-asf.ponee.io@openwhisk.apache.org Fri Apr 6 20:48:09 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 09E05180649 for ; Fri, 6 Apr 2018 20:48:08 +0200 (CEST) Received: (qmail 90850 invoked by uid 500); 6 Apr 2018 18:48:08 -0000 Mailing-List: contact commits-help@openwhisk.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@openwhisk.apache.org Delivered-To: mailing list commits@openwhisk.apache.org Received: (qmail 90841 invoked by uid 99); 6 Apr 2018 18:48:08 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Apr 2018 18:48:08 +0000 Received: by gitbox.apache.org (ASF Mail Server at gitbox.apache.org, from userid 33) id 8BBA18505F; Fri, 6 Apr 2018 18:48:07 +0000 (UTC) Date: Fri, 06 Apr 2018 18:48:07 +0000 To: "commits@openwhisk.apache.org" Subject: [incubator-openwhisk-deploy-kube] branch master updated: Test on multiple Kubernetes versions (#177) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Message-ID: <152304048751.16280.10595478184704058200@gitbox.apache.org> From: dgrove@apache.org X-Git-Host: gitbox.apache.org X-Git-Repo: incubator-openwhisk-deploy-kube X-Git-Refname: refs/heads/master X-Git-Reftype: branch X-Git-Oldrev: 74ad1d2838b494835be95a8d44f8f8db94a955ad X-Git-Newrev: 2eb9aa4dc15d9e83a9871b307f20d79fb74be9e9 X-Git-Rev: 2eb9aa4dc15d9e83a9871b307f20d79fb74be9e9 X-Git-NotificationType: ref_changed_plus_diff X-Git-Multimail-Version: 1.5.dev Auto-Submitted: auto-generated This is an automated email from the ASF dual-hosted git repository. dgrove pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-deploy-kube.git The following commit(s) were added to refs/heads/master by this push: new 2eb9aa4 Test on multiple Kubernetes versions (#177) 2eb9aa4 is described below commit 2eb9aa4dc15d9e83a9871b307f20d79fb74be9e9 Author: David Grove AuthorDate: Fri Apr 6 14:48:03 2018 -0400 Test on multiple Kubernetes versions (#177) Expand matrix to test multiple Kubernetes versions and minikube versions. Expand warning on bad minor versions of Kubernetes to avoid due to Kubernetes regressions. Default OW_CONTAINER_FACTORY to docker if not set to preserve simple usage of build.sh to deploy default configuration to minikube. More precise `kubectl get pods` command using labels to select pods in build.sh to avoid spurious matches on health containers for KubernetesContainerFactory. Add troubleshooting note about minikube DNS problems and likely workaround. --- .travis.yml | 8 ++++---- README.md | 24 ++++++++++++++++-------- docs/setting_up_minikube/README.md | 4 ++++ tools/travis/build.sh | 16 ++++++++++------ 4 files changed, 34 insertions(+), 18 deletions(-) diff --git a/.travis.yml b/.travis.yml index 584f4bb..7b38ed4 100644 --- a/.travis.yml +++ b/.travis.yml @@ -2,13 +2,13 @@ sudo: required env: global: - - TRAVIS_KUBE_VERSION=v1.7.4 - - TRAVIS_MINIKUBE_VERSION=v0.23.0 - secure: d7CuMXbhT83W2x78qiLwgogX1+3aPicd1PlTwwNNDN6QSkImbxareyKThnsqlHIiNj3o5l5DBuiYjy7wrF/xD1g8BQMmTwm99DRx5q3CI3Im3VCi/ZK8SaNjuOy24d7cf5k2tB/87Gk7zmKsMDYm+fpCl+GpgUmIEeIwthiAxuXSDWZ8eQPIptmxj56DeFRNouvXG+dEUtBfWiwN27UPxNKExCixFnegmdtffLbz6hhst7BHr5Ry9acbycre98PCwWZcu9lxFs+SJ1kvnzX2iue4otmDkF1WkJjxaOFPJVs/D3YItg+neLCSxjwBskPed+Fct8bOjcM/uVROJPNIq5icBmaPX2isH0lvtxOeVw/dmioWYXYPN9ygBOe4eO/vtPllN0bcAUo5xl9jXev8ciAozYrYpHVh9Fplfd81rcYTeYzALmRJBdoiWoc3KQGzwGc9sB1ffmy+KWgG9T0zbnS4fALSR4PS [...] - secure: CJtnU94HTDqd4A6uvhFl8IpnmU+wTdlzb8bPBFUl/lI/VKXiRrYpgJdKUro5xEoxFKuqMprLhbyf66niyWLTIeogjUAEu/h/o2dBVeGgSGKoqC0hQgqvnxKFeGlzFJ0XuEs3vbStJGRnQszGsfnnDrscJtR0x9X+1w4aBKI7iPyyuFtVkDD1UsmBbSi+M8FTeq7G7A0reMDaey7uog3CFCpIMl4geshcohQEcKEGbnXQZoLPFpb7cBOE83VXBJ7Y7Dgf/U4keiLovvnuJThGKZm/SVV2KlELmBmtmbx3rMT6Vb5k9ChSdRWapromNnnzmJBIQ5Scc2mwV3A93/SMha1F3IlYpDKs5djfTw8jZfVnuiou7HhTaRjHkmmcwP12/k30gLe2kw0Vezg1TCY4zgtOpcmCxc8RHEy0ceA74rKvRi8LbexTCwX+iAMQFn/pSrh/OqAq/50JbLyczcoO1zXWS38txUQN [...] matrix: - - OW_CONTAINER_FACTORY=docker - - OW_CONTAINER_FACTORY=kube + - OW_CONTAINER_FACTORY=docker TRAVIS_KUBE_VERSION=v1.7.4 TRAVIS_MINIKUBE_VERSION=v0.23.0 + - OW_CONTAINER_FACTORY=docker TRAVIS_KUBE_VERSION=v1.8.0 TRAVIS_MINIKUBE_VERSION=v0.25.2 + - OW_CONTAINER_FACTORY=docker TRAVIS_KUBE_VERSION=v1.9.0 TRAVIS_MINIKUBE_VERSION=v0.25.2 + - OW_CONTAINER_FACTORY=kube TRAVIS_KUBE_VERSION=v1.8.0 TRAVIS_MINIKUBE_VERSION=v0.25.2 services: - docker diff --git a/README.md b/README.md index cc701e5..a9f4501 100644 --- a/README.md +++ b/README.md @@ -17,20 +17,28 @@ This repository can be used to deploy OpenWhisk to a Kubernetes cluster. Several requirements must be met for OpenWhisk to deploy on Kubernetes. **Kubernetes** -* [Kubernetes](https://github.com/kubernetes/kubernetes) version 1.6+. However, avoid Kubernetes 1.6.3 due to an [issue with volume mount subpaths](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.6.md#known-issues-for-v163). Our Travis CI testing uses Kubernetes version 1.7.4. +* [Kubernetes](https://github.com/kubernetes/kubernetes) version 1.6+. However, multiple minor releases of Kubernetes, including 1.6.3, 1.7.14, 1.8.9 and 1.9.4 will not work for OpenWhisk due to bugs with volume mount subpaths (see[[1](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.6.md#known-issues-for-v163), [2](https://github.com/kubernetes/kubernetes/issues/61076)]. This bug will surface as a failure when deploying the nginx container. * The ability to create Ingresses to make a Kubernetes service available outside of the cluster so you can actually use OpenWhisk. -* Endpoints of Kubernetes services must be able to loopback to themselves ("hairpin mode"). - -**OpenWhisk** -* Docker version 1.12+ +* Endpoints of Kubernetes services must be able to loopback to themselves (the kubelet's `hairpin-mode` must not be `none`). # Setting up Kubernetes ## Using Minikube -For local development and testing, we recommend using Minikube version 0.23+ -with the docker network in promiscuous mode. Our Travis CI testing uses Minikube 0.23.0. -Take a look at these [instructions](/docs/setting_up_minikube/README.md). +For local development and testing, we recommend using Minikube with +the docker network in promiscuous mode. Not all combinations of +Minikube and Kubernetes versions will work for running OpenWhisk. +Although other combinations may work, we recommend at least initially +using a combination from the table below that is verified by our +Travis CI testing. + +| Kubernetes Version | Minikube Version | +--- | --- | +1.7.4 | 0.23.0 | +1.8.0 | 0.25.2 | +1.9.0 | 0.25.2 | + +For details on setting up Minikube, see these [instructions](/docs/setting_up_minikube/README.md). ## Using a Kubernetes cluster from a cloud provider diff --git a/docs/setting_up_minikube/README.md b/docs/setting_up_minikube/README.md index 9ba6b0a..805ea8c 100644 --- a/docs/setting_up_minikube/README.md +++ b/docs/setting_up_minikube/README.md @@ -45,3 +45,7 @@ minikube ssh -- sudo ip link set docker0 promisc on ``` Your Minikube cluster should now be ready to deploy OpenWhisk. + +# Troubleshooting + +For some combinations of Minikube and Kubernetes versions, you may need to workaround a [Minikube DNS issue](https://github.com/kubernetes/minikube/issues/2240#issuecomment-348319371). A common symptom of this issue is that the OpenWhisk couchdb pod will fail to start with the error that it is unable to resolve `github.com` when cloning the openwhisk git repo. A work around is to delete the minikube cluster, issue the command `minikube config set bootstrapper kubeadm` and then redo the ` [...] diff --git a/tools/travis/build.sh b/tools/travis/build.sh index 0521d6e..e59c8d8 100755 --- a/tools/travis/build.sh +++ b/tools/travis/build.sh @@ -7,7 +7,7 @@ couchdbHealthCheck () { # wait for the pod to be created before getting the job name sleep 5 - POD_NAME=$(kubectl -n openwhisk get pods -o wide --show-all | grep "couchdb" | awk '{print $1}') + POD_NAME=$(kubectl -n openwhisk get pods -l name=couchdb -o wide --show-all | grep "couchdb" | awk '{print $1}') PASSED=false TIMEOUT=0 @@ -40,7 +40,7 @@ deploymentHealthCheck () { PASSED=false TIMEOUT=0 until $PASSED || [ $TIMEOUT -eq 60 ]; do - KUBE_DEPLOY_STATUS=$(kubectl -n openwhisk get pods -o wide | grep "$1" | awk '{print $3}') + KUBE_DEPLOY_STATUS=$(kubectl -n openwhisk get pods -l name="$1" -o wide | grep "$1" | awk '{print $3}') if [ "$KUBE_DEPLOY_STATUS" == "Running" ]; then PASSED=true break @@ -55,7 +55,7 @@ deploymentHealthCheck () { if [ "$PASSED" = false ]; then echo "Failed to finish deploying $1" - kubectl -n openwhisk logs $(kubectl -n openwhisk get pods -o wide | grep "$1" | awk '{print $1}') + kubectl -n openwhisk logs $(kubectl -n openwhisk get pods -l name="$1" -o wide | grep "$1" | awk '{print $1}') exit 1 fi @@ -71,7 +71,7 @@ statefulsetHealthCheck () { PASSED=false TIMEOUT=0 until $PASSED || [ $TIMEOUT -eq 60 ]; do - KUBE_DEPLOY_STATUS=$(kubectl -n openwhisk get pods -o wide | grep "$1"-0 | awk '{print $3}') + KUBE_DEPLOY_STATUS=$(kubectl -n openwhisk get pods -l name="$1" -o wide | grep "$1"-0 | awk '{print $3}') if [ "$KUBE_DEPLOY_STATUS" == "Running" ]; then PASSED=true break @@ -135,6 +135,9 @@ set -x SCRIPTDIR=$(cd $(dirname "$0") && pwd) ROOTDIR="$SCRIPTDIR/../../" +# Default to docker container factory if not specified +OW_CONTAINER_FACTORY=${OW_CONTAINER_FACTORY:="docker"} + cd $ROOTDIR # Label invoker nodes (needed for DockerContainerFactory-based invoker deployment) @@ -220,18 +223,19 @@ pushd kubernetes/invoker echo "Deploying invoker using DockerContainerFactory" kubectl -n openwhisk create cm invoker.config --from-env-file=invoker-dcf.env kubectl apply -f invoker-dcf.yml - deploymentHealthCheck "invoker" elif [ "$OW_CONTAINER_FACTORY" = "kube" ]; then echo "Deploying invoker using KubernetesContainerFactory" kubectl -n openwhisk create cm invoker.config --from-env-file=invoker-k8scf.env kubectl apply -f invoker-agent.yml deploymentHealthCheck "invoker-agent" kubectl apply -f invoker-k8scf.yml - deploymentHealthCheck "invoker-0" else echo "Unknown container factory $OW_CONTAINER_FACTORY" exit 1 fi + + # wait until invoker is ready + deploymentHealthCheck "invoker" popd # setup nginx -- To stop receiving notification emails like this one, please contact dgrove@apache.org.