openwhisk-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mrutkow...@apache.org
Subject [incubator-openwhisk-deploy-kube] branch master updated: reorganize docs describing Kubernetes options and mycluster.yaml (#335)
Date Tue, 06 Nov 2018 21:39:57 GMT
This is an automated email from the ASF dual-hosted git repository.

mrutkowski pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-deploy-kube.git


The following commit(s) were added to refs/heads/master by this push:
     new 4f0b85c  reorganize docs describing Kubernetes options and mycluster.yaml (#335)
4f0b85c is described below

commit 4f0b85c7fe410c4ac0fadb8b9ef471a951a20fa0
Author: David Grove <dgrove-oss@users.noreply.github.com>
AuthorDate: Tue Nov 6 16:39:52 2018 -0500

    reorganize docs describing Kubernetes options and mycluster.yaml (#335)
    
    * reorganize docs describing Kubernetes options and mycluster.yaml
    
    * Expand dind documentation
    
    * expand minikube docs
    
    * expand docker-for-mac docs
---
 README.md                             | 226 ++++++++++++++++++----------------
 docs/ingress.md                       |   8 +-
 docs/k8s-dind-cluster.md              | 116 +++++++++++++++++
 docs/k8s-docker-for-mac.md            |  94 ++++++++++++++
 docs/{minikube.md => k8s-minikube.md} |  71 ++++++++---
 docs/k8s-technical-requirements.md    |  27 ++++
 6 files changed, 417 insertions(+), 125 deletions(-)

diff --git a/README.md b/README.md
index a9b88f8..47f27cb 100644
--- a/README.md
+++ b/README.md
@@ -23,109 +23,76 @@
 [![Build Status](https://travis-ci.org/apache/incubator-openwhisk-deploy-kube.svg?branch=master)](https://travis-ci.org/apache/incubator-openwhisk-deploy-kube)
 [![Join Slack](https://img.shields.io/badge/join-slack-9B69A0.svg)](http://slack.openwhisk.org/)
 
-This repository can be used to deploy OpenWhisk to a Kubernetes cluster.
+This repository can be used to deploy OpenWhisk to Kubernetes.
+It contains Helm charts, documentation, and supporting
+configuration files and scripts that can be used to deploy OpenWhisk
+to both single-node and multi-node Kubernetes clusters.
 
 # Table of Contents
 
-* [Setting up Kubernetes and Helm](#setting-up-kubernetes-and-helm)
+* [Prerequisites: Kubernetes and Helm](#prerequisites-kubernetes-and-helm)
 * [Deploying OpenWhisk](#deploying-openwhisk)
 * [Deploying OpenWhisk Providers](#deploying-openwhisk-providers)
 * [Development and Testing](#development-and-testing)
 * [Cleanup](#cleanup)
 * [Issues](#issues)
 
-# Setting up Kubernetes and Helm
+# Prerequisites: Kubernetes and Helm
 
-## Kubernetes
-
-### Requirements
-
-Several requirements must be met for OpenWhisk to deploy on Kubernetes.
-* [Kubernetes](https://github.com/kubernetes/kubernetes) version 1.9+. However, version 1.9.4
will not work for OpenWhisk due to a bug with volume mount subpaths (see[[kubernetes-61076](https://github.com/kubernetes/kubernetes/issues/61076)]).
This bug will surface as a failure when deploying the nginx container.
-* The ability to create Ingresses to make a Kubernetes service available outside of the cluster
so you can actually use OpenWhisk.
-* If you enable persistence (see [docs/configurationChoices.md](./docs/configurationChoices.md)),
either your cluster is configured to support [Dynamic Volume Provision](https://kubernetes.io/docs/concepts/storage/dynamic-provisioning/)
or you must manually create any necessary PersistentVolumes when deploying the Helm chart.
-* Endpoints of Kubernetes services must be able to loopback to themselves (the kubelet's
`hairpin-mode` must not be `none`).
-
-### Using Kubernetes in Docker for Mac
-
-If you are using a Mac as your development machine, the simplest way
-to get a Kubernetes cluster for local development is to use the
-built-in support for running a single node Kubernetes cluster that is
-available in Docker 18.06 and later.  This will let you use Helm to
-deploy Apache OpenWhisk to Kubernetes on your laptop without needing
-to install Minikube or otherwise run inside a virtual machine.
-
-Step-by-step instructions on enabling Kubernetes in Docker are
-available as part of the
-[Getting started](https://docs.docker.com/docker-for-mac/#kubernetes)
-documentation from Docker.
-
-In a nutshell, open the Docker preferences window, switch to the
-`Advanced` panel and make sure you have at least 4GB of Memory
-allocated to Docker. Then switch to the Kubernetes panel, and check
-the box to enable Kubernetes. It is recommended that you use the
-`kubectl` cli that is installed by Docker in `/usr/local/bin`, so
-please make sure it is appears in your path before any `kubectl` you
-might also have installed on your machine.  Finally, pick the
-`docker-for-desktop` config for `kubectl` by executing the command
-`kubectl config use-context docker-for-desktop`.
-
-One nice feature of using Kubernetes in Docker, is that the
-containers being run in Kubernetes are also directly
-visible/accessible via the usual Docker commands. Furthermore, it is
-straightforward to deploy local images by adding a stanza to your
-mycluster.yaml. For example, to use a locally built controller image,
-just add the stanza below to your `mycluster.yaml` to override the default
-behavior of pulling `openwhisk/controller:latest` from Docker Hub.
-```yaml
-controller:
-  image: "whisk/controller"
-  imagePullPolicy: "IfNotPresent"
-```
-
-NOTE: Docker for Windows 18.06 and later has similar built-in support
-for Kubernetes. We would be interested in any experience using it to
-run Apache OpenWhisk on the Windows platform.
+[Kubernetes](https://kubernetes.io/) is a container orchestration
+platform that automates the deployment, scaling, and management of
+containerized applications. [Helm](https://helm.sh/) is a package
+manager for Kubernetes that simplifies the management of Kubernetes
+applications. You do not need to be an expert on either Kubernetes or
+Helm to use this project, but you may find it useful to review their
+overview documentation at the links above to become familiar with
+their key concepts and terminology.
 
-### Using kubeadm-dind-cluster
-On Linux, you can get a similar experience to using Kubernetes in
-Docker for Mac via the
-[kubeadm-dind-cluster](https://github.com/kubernetes-sigs/kubeadm-dind-cluster)
-project.  In a nutshell, you can get started by doing
-```shell
-# Get the script for the Kubernetes version you want
-wget https://cdn.rawgit.com/kubernetes-sigs/kubeadm-dind-cluster/master/fixed/dind-cluster-v1.10.sh
-chmod +x dind-cluster-v1.10.sh
-
-# start the cluster. Please note you *must* set `USE_HAIRPIN` to `true`
-USE_HAIRPIN=true ./dind-cluster-v1.10.sh up
-
-# add kubectl directory to your PATH
-export PATH="$HOME/.kubeadm-dind-cluster:$PATH"
-```
+## Kubernetes
 
-Our TravisCI testing uses kubeadm-dind-cluster.sh on an ubuntu 16.04
-host.  The `fixed` `dind-cluster` scripts for Kubernetes version 1.10
-and 1.11 are known to work for deploying OpenWhisk.
+Your first step is to create a Kubernetes cluster that is capable of
+supporting an OpenWhisk deployment. Although there are some [technical
+requirements](docs/k8s-technical-requirements.md) that the Kubernetes
+cluster must satisfy, any of the options described below is
+acceptable.
+
+### Simple Docker-based options
+
+The simplest way to get a small Kubernetes cluster suitable for
+development and testing is to use one of the Docker-in-Docker
+approaches for running Kubernetes directly on top of Docker on your
+development machine. Depending on your host operating system, we
+recommend the following:
+1. MacOS: Use the built-in Kubernetes support in Docker for Mac
+version 18.06 or later. Please follow our
+[setup instructions](docs/k8s-docker-for-mac.md) to initially create
+your cluster.
+2. Linux: Use kubeadm-dind-cluster, but carefully follow our
+[setup instructions](docs/k8s-dind-cluster.md) because the default
+setup of kubeadm-dind-cluster does *not* meet the requirements for
+running OpenWhisk.
+3. Windows: We believe that just like with MacOS, the built-in
+Kubernetes support in Docker for Windows version 18.06 or later should
+be sufficient to run OpenWhisk.  We would welcome a pull request with
+provide detailed setup instructions for Windows.
 
 ### Using Minikube
 
-If you are on Linux and do not want to use kubeadm-dind-cluster, then
-an alternative for local development and testing, is using Minikube
-with the docker network in promiscuous mode.  However not all
-combinations of Minikube and Kubernetes versions will work for running
-OpenWhisk. Some known good combinations are:
-
-| Kubernetes Version | Minikube Version |
---- | --- |
-1.9.0 | 0.25.2 |
-1.10.5 | 0.28.2 |
-
-For details on setting up Minikube, see these [instructions](docs/minikube.md).
+Minikube provides a Kubernetes cluster running inside a virtual
+machine (for example VirtualBox). It can be used on MacOS, Linux, or
+Windows to run OpenWhisk, but is somewhat more finicky than the
+docker-in-docker options described above. For details on setting up
+Minikube, see these [instructions](docs/k8s-minikube.md).
 
 ### Using a Kubernetes cluster from a cloud provider
 
-You can also provision a Kubernetes cluster from a cloud provider, subject to the cluster
meeting the requirements above.
+You can also provision a Kubernetes cluster from a cloud provider,
+subject to the cluster meeting the [technical
+requirements]([docs/k8s-technical-requirements.md).  Managed
+Kubernetes services from IBM (IKS), Google (GKE), and Amazon (EKS) are
+known to work for running OpenWhisk and are all documented and
+supported by this project.  We would welcome contributions of
+documentation for Azure (AKS) and any other public cloud providers.
 
 ## Helm
 
@@ -147,6 +114,9 @@ kubectl create clusterrolebinding tiller-cluster-admin --clusterrole=cluster-adm
 
 # Deploying OpenWhisk
 
+Now that you have your Kubernetes cluster and have installed and
+initialized Helm, you are ready to deploy OpenWhisk.
+
 ## Overview
 
 You will use Helm to deploy OpenWhisk to your Kubernetes cluster.
@@ -187,34 +157,76 @@ $ kubectl label nodes <INVOKER_NODE_NAME> openwhisk-role=invoker
 
 ## Customize the Deployment
 
-You will need to create a mycluster.yaml file that records how the
-OpenWhisk deployment on your cluster will be accessed by clients.  See
-the [ingress discussion](./docs/ingress.md) for details. Below is a sample
-file appropriate for a Minikube cluster where `minikube ip` returns
-`192.168.99.100` and port 31001 is available to be used.  If you are
-using Docker for Mac, you can use the same configuration but use the
-command `kubectl describe nodes | grep InternalIP` to determine the
-value for `apiHostName`.  If you are using kubeadm-dind-cluster, use
-the command `kubectl describe node kube-node-2 | grep InternalIP` to
-determine the value for `apiHostName`.
+You must create a `mycluster.yaml` file to record key aspects of your
+Kubernetes cluster that are needed to configure the deployment of
+OpenWhisk to your cluster. Most of the needed configuration is related
+to networking and is described in the [ingress discussion](./docs/ingress.md).
+
+Beyond specifying the ingress, the `mycluster.yaml` file is also used
+to customize your OpenWhisk deployment by enabling optional features
+and controlling the replication factor of the various microservices
+that make up the OpenWhisk implementation. See the [configuration
+choices documentation](./docs/configurationChoices.md) for a
+discussion of the primary options.
+
+### Sample mycluster.yaml for Docker for Mac
 
+Here is a sample file for a Docker for Mac deployment where
+`kubectl describe nodes | grep InternalIP` returns 192.168.65.3 and port 31001 is available
to
+be used on your host machine.
 ```yaml
 whisk:
   ingress:
     type: NodePort
-    apiHostName: 192.168.99.100
+    apiHostName: 192.168.65.3
     apiHostPort: 31001
 
 nginx:
   httpsNodePort: 31001
 ```
 
-Beyond specifying the ingress, the `mycluster.yaml` file is also used
-to customize your OpenWhisk deployment by enabling optional features
-and controlling the replication factor of the various microservices
-that make up the OpenWhisk implementation. See the [configuration
-choices documentation](./docs/configurationChoices.md) for a
-discussion of the primary options.
+### Sample mycluster.yaml for kubeadm-dind-cluster.sh
+
+Here is a sample file for a kubeadm-dind-cluster where `kubectl describe node kube-node-1
|
+grep InternalIP` returns 10.192.0.3 and port 31001 is available to
+be used on your host machine.
+```yaml
+whisk:
+  ingress:
+    type: NodePort
+    apiHostName: 10.192.0.3
+    apiHostPort: 31001
+
+nginx:
+  httpsNodePort: 31001
+
+invoker:
+  containerFactory:
+    dind: true
+```
+
+Note the stanza setting `invoker.containerFactory.dind` to `true`.
+This stanza is required; failure to override the default of `false` inherited
+from `helm/openwhisk/values.yaml` will result in a deployment
+of OpenWhisk with no healthy invokers (and thus a deployment that
+will not execute any user actions).
+
+### Sample mycluster.yaml for Minikube
+
+Here is a sample file appropriate for a Minikube cluster where
+`minikube ip` returns `192.168.99.100` and port 31001 is available to
+be used on your host machine.
+
+```yaml
+whisk:
+  ingress:
+    type: NodePort
+    apiHostName: 192.168.99.100
+    apiHostPort: 31001
+
+nginx:
+  httpsNodePort: 31001
+```
 
 ## Deploy With Helm
 
@@ -323,11 +335,11 @@ launch the tests:
 
 ### Deploying a locally built docker image.
 
-By overriding the default `image` and `imagePullPolicy` for one or
-more OpenWhisk components, you can run locally built docker images.
-For example, to use a locally built controller image, just add the
-stanza below to your `mycluster.yaml` to override the default behavior
-of pulling `openwhisk/controller:latest` from Docker Hub.
+If you are using Kubernetes in Docker, it is
+straightforward to deploy local images by adding a stanza to your
+mycluster.yaml. For example, to use a locally built controller image,
+just add the stanza below to your `mycluster.yaml` to override the default
+behavior of pulling `openwhisk/controller:latest` from Docker Hub.
 ```yaml
 controller:
   image: "whisk/controller"
diff --git a/docs/ingress.md b/docs/ingress.md
index f95aea7..34f26f0 100644
--- a/docs/ingress.md
+++ b/docs/ingress.md
@@ -92,10 +92,10 @@ nginx:
 
 Obtain the IP address of one of the two Kubernetes worker nodes using
 the command below.  Although the nginx NodePort service is actually
-available on both of the nodes, by using the node which you did not
-label with `openwhisk-role=invoker` as your api-host you can cut 1 hop
-out of the network path. So, if you label `kube-node-2` as your
-invoker node, pick `kube-node-1` as your api_host.
+available on both of the nodes, by using the node which you labelled
+with `openwhisk-role=core` as your api-host you can cut 1 hop
+out of the network path. So, if you label `kube-node-1` as your
+core node, pick `kube-node-1` as your api_host.
 ```shell
 kubectl describe node kube-node-1 | grep InternalIP
 ```
diff --git a/docs/k8s-dind-cluster.md b/docs/k8s-dind-cluster.md
new file mode 100644
index 0000000..ec45911
--- /dev/null
+++ b/docs/k8s-dind-cluster.md
@@ -0,0 +1,116 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+
+# Using kubeadm-dind-cluster for OpenWhisk
+
+## Overview
+
+On Linux, you can run Kubernetes on top of Docker using the
+[kubeadm-dind-cluster](https://github.com/kubernetes-sigs/kubeadm-dind-cluster)
+project.  Based on using Docker-in-Docker (DIND) virtualization and
+`kubeadm`, kubeadm-dind-cluster can be used to create a
+multi-node Kubernetes cluster that is suitable for deploying
+OpenWhisk for development and testing.  For detailed instructions on kubeadm-dind-cluster,
we
+refer you to that project's [github repository](https://github.com/kubernetes-sigs/kubeadm-dind-cluster).
+Here we will only cover the basic operations needed to create and
+operate a default cluster with two virtual worker nodes running on a
+single host machine.
+
+## Initial setup
+
+There are "fixed" scripts
+[available](https://github.com/kubernetes-sigs/kubeadm-dind-cluster/tree/master/fixed)
+for each major release of Kubernetes.
+Our TravisCI testing uses kubeadm-dind-cluster.sh on an ubuntu 16.04
+host.  The `fixed` `dind-cluster` scripts for Kubernetes version 1.10
+and 1.11 are known to work for deploying OpenWhisk.
+
+### Creating the Kubernetes Cluster
+
+First, make sure your userid is in the `docker` group on the host
+machine.  This will enable you to run `dind-cluster.sh` script without
+requiring `sudo` to gain `root` privileges.
+
+To initially create your cluster, do the following:
+```shell
+# Get the script for the Kubernetes version you want
+wget https://cdn.rawgit.com/kubernetes-sigs/kubeadm-dind-cluster/master/fixed/dind-cluster-v1.10.sh
+
+# Make it executable
+chmod +x dind-cluster-v1.10.sh
+
+# Start the cluster. Please note you *must* set `USE_HAIRPIN` to `true`
+USE_HAIRPIN=true ./dind-cluster-v1.10.sh up
+
+# add the directory containing kubectl to your PATH
+export PATH="$HOME/.kubeadm-dind-cluster:$PATH"
+```
+
+The default configuration of `dind-cluster.sh` will create a cluster
+with three nodes: 1 master node and two worker nodes. We recommend
+labeling the two worker nodes for OpenWhisk so that you have 1 invoker
+node for running user actions and 1 core node for running the rest of
+the OpenWhisk system.
+```shell
+kubectl label node kube-worker-1 openwhisk-role=core
+kubectl label node kube-worker-2 openwhisk-role=invoker
+```
+
+### Configuring OpenWhisk
+
+Because the container logs for docker containers running on the
+virtual worker nodes are in a non-standard location, you must
+configure the invoker to look for user action logs in a different
+path. You do that by adding the following required stanza to your
+mycluster.yaml.
+```yaml
+invoker:
+  containerFactory:
+    dind: true
+```
+
+You will be using a NodePort ingress to access OpenWhisk. Assuming
+w`kubectl describe node kube-node-1 | grep InternalIP` returns 10.192.0.3
+and port 31001 is available to be used on your host machine, you can
+add the following stanzas of to your mycluster.yaml:
+```yaml
+whisk:
+  ingress:
+    type: NodePort
+    apiHostName: 10.192.0.3
+    apiHostPort: 31001
+
+nginx:
+  httpsNodePort: 31001
+```
+
+## Limitations
+
+Using kubeadm-dind-cluster is only appropriate for development and
+testing purposes.  It is not recommended for production deployments of
+OpenWhisk.
+
+Unlike using Kubernetes with Docker for Mac 18.06 and later, only the
+virtual master/worker nodes are visible to Docker on the host system. The
+individual pods running the OpenWhisk system are only visible using
+`kubectl` and not directly via host Docker commands.
+
+There does not appear to be a reliable way to restart the Kubernetes
+cluster without also re-installing Helm and OpenWhisk.
diff --git a/docs/k8s-docker-for-mac.md b/docs/k8s-docker-for-mac.md
new file mode 100644
index 0000000..0b37480
--- /dev/null
+++ b/docs/k8s-docker-for-mac.md
@@ -0,0 +1,94 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+# Using Kubernetes in Docker for Mac for OpenWhisk
+
+## Overview
+
+If you are using a Mac as your development machine, the simplest way
+to get a Kubernetes cluster for local development is to use the
+built-in support for running a single node Kubernetes cluster that is
+available in Docker 18.06 and later.  This will let you use Helm to
+deploy Apache OpenWhisk to Kubernetes on your laptop without needing
+to install Minikube or otherwise run inside a virtual machine.
+
+## Initial setup
+
+### Creating the Kubernetes Cluster
+
+Step-by-step instructions on enabling Kubernetes in Docker are
+available as part of the
+[Getting started](https://docs.docker.com/docker-for-mac/#Kubernetes)
+documentation from Docker.
+
+In a nutshell, open the Docker preferences window, switch to the
+`Advanced` panel and make sure you have at least 4GB of Memory
+allocated to Docker. Then switch to the Kubernetes panel, and check
+the box to enable Kubernetes. It is recommended that you use the
+`kubectl` cli that is installed by Docker in `/usr/local/bin`, so
+please make sure it is appears in your path before any `kubectl` you
+might also have installed on your machine.  Finally, pick the
+`docker-for-desktop` config for `kubectl` by executing the command
+`kubectl config use-context docker-for-desktop`.
+
+### Configuring OpenWhisk
+
+You will be using a NodePort ingress to access OpenWhisk. Assuming
+`kubectl describe nodes | grep InternalIP` returns 192.168.65.3 and
+port 31001 is available to be used on your host machine, you can add
+the following stanzas of to your mycluster.yaml:
+```yaml
+whisk:
+  ingress:
+    type: NodePort
+    apiHostName: 192.168.65.3
+    apiHostPort: 31001
+
+nginx:
+  httpsNodePort: 31001
+```
+
+## Hints and Tips
+
+One nice feature of using Kubernetes in Docker, is that the
+containers being run in Kubernetes are also directly
+visible/accessible via the usual Docker commands. Furthermore, it is
+straightforward to deploy local images by adding a stanza to your
+mycluster.yaml. For example, to use a locally built controller image,
+just add the stanza below to your `mycluster.yaml` to override the default
+behavior of pulling `openwhisk/controller:latest` from Docker Hub.
+```yaml
+controller:
+  image: "whisk/controller"
+  imagePullPolicy: "IfNotPresent"
+```
+
+## Limitations
+
+Using Kubernetes in Docker for Mac is only appropriate for development
+and testing purposes.  It is not recommended for production
+deployments of OpenWhisk.
+
+The docker network is not exposed to the host on MacOS. However, the
+exposed ports for NodePort services are forwarded from localhost.
+Therefore you must use different host names to connect to OpenWhisk
+from outside the cluster (with the `wsk` cli) and from inside the
+cluster (in `mycluster.yaml`).  Continuing the example from above,
+when setting the `--apihost` for the `wsk` cli, you would use
+`localhost:31001`.
diff --git a/docs/minikube.md b/docs/k8s-minikube.md
similarity index 60%
rename from docs/minikube.md
rename to docs/k8s-minikube.md
index 3462e4e..3895fff 100644
--- a/docs/minikube.md
+++ b/docs/k8s-minikube.md
@@ -17,7 +17,19 @@
 #
 -->
 
-# Setting Up Minikube for OpenWhisk
+# Using Minikube for OpenWhisk
+
+## Overview
+
+Minikube runs a single node Kubernetes cluster inside of a VM
+(virtual machine) running on your host machine. Depending on your host
+machine, you will have different choices for the VM.
+Minikube is suitable for developing and testing of OpenWhisk.  Because
+the Kubernetes cluster is run within another VM, Minikube is slightly
+more resource intensive and slightly less flexible than some of the
+other docker-in-docker based approaches to running Kubernetes.
+
+## Initial Minikube Setup
 
 New versions of Minikube and Kubernetes are released fairly
 frequently.  Over time, you will probably end up needing to have
@@ -25,9 +37,9 @@ multiple versions installed on your development machine. We recommend
 using the asdf package manager to make it very easy to switch between
 versions and manage installation.
 
-## Install and configure asdf
+### Install and configure asdf
 
-### MacOS
+#### MacOS
 ```
 brew install asdf
 ```
@@ -37,20 +49,22 @@ Edit your `~/.profile` or equivalent
 [ -s "/usr/local/opt/asdf/asdf.sh" ] && . /usr/local/opt/asdf/asdf.sh
 ```
 
-### Other Platforms
+#### Other Platforms
 
 Follow the asdf installation instructions at https://github.com/asdf-vm/asdf
 
-## Add minikube and kubectl plugins
+### Add minikube and kubectl plugins
 ```
 asdf plugin-add kubectl
 asdf plugin-add minikube
 ```
 
-## Install minikube and kubectl using asdf.
-We recommend starting with versions of minikube and kubectl that match
-what we test in TravisCI. After you have experience with OpenWhisk on
-Minikube, feel free to experiment with additional versions.
+### Install minikube and kubectl using asdf.
+
+We recommend starting a combination of minikube 0.28.2 and kubectl
+1.10.5 that are known to work for running OpenWhisk.  After you have
+experience with OpenWhisk on Minikube, feel free to experiment with
+additional versions.
 
 ```
 asdf install kubectl 1.10.5
@@ -59,7 +73,7 @@ asdf install minikube 0.28.2
 asdf global minikube 0.28.2
 ```
 
-## Configure the Minikube VM
+### Configure the Minikube VM
 
 You will want at least 4GB of memory and 2 CPUs for Minikube to run OpenWhisk.
 If you have a larger machine, you may want to provision more (especially more memory).
@@ -71,7 +85,7 @@ minikube config set memory 4096
 minikube config set WantUpdateNotification false
 ```
 
-## Start Minikube
+### Start Minikube
 
 With minikube v0.25.2:
 ```
@@ -82,8 +96,7 @@ with minikube versions more recent than v0.25.2:
 minikube start
 ```
 
-
-## Setup Docker network in promiscuous mode
+### Setup Docker network in promiscuous mode
 Put the docker network in promiscuous mode.
 ```
 minikube ssh -- sudo ip link set docker0 promisc on
@@ -93,7 +106,7 @@ minikube ssh -- sudo ip link set docker0 promisc on
 
 Your Minikube cluster should now be ready to deploy OpenWhisk.
 
-## Changing Kubernetes versions
+### Changing Kubernetes versions
 
 To use a different version of Kubernetes with Minikube, you need to delete the VM, reconfigure
minikube, restart, and
 redo the setup of the Docker network.
@@ -103,3 +116,33 @@ minikube config set kubernetes-version <NEW_VERSION>
 minikube start [--extra-config=apiserver.Authorization.Mode=RBAC]
 minikube ssh -- sudo ip link set docker0 promisc on
 ```
+
+## Configuring OpenWhisk
+
+You will be using a NodePort ingress to access OpenWhisk. Assuming
+`minikube ip` returns `192.168.99.100` and port 31001 is available to
+be used on your host machine,  you can add the following stanzas of to
+your mycluster.yaml:
+
+```yaml
+whisk:
+  ingress:
+    type: NodePort
+    apiHostName: 192.168.99.100
+    apiHostPort: 31001
+
+nginx:
+  httpsNodePort: 31001
+```
+
+## Limitations
+
+Using Minikube is only appropriate for development and testing
+purposes.  It is not recommended for production deployments of
+OpenWhisk.
+
+You must remember to put the docker network in promiscuous mode via
+```
+minikube ssh -- sudo ip link set docker0 promisc on
+```
+every time you start minikube.
diff --git a/docs/k8s-technical-requirements.md b/docs/k8s-technical-requirements.md
new file mode 100644
index 0000000..04138db
--- /dev/null
+++ b/docs/k8s-technical-requirements.md
@@ -0,0 +1,27 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+
+# Technical Requirements for Kubernetes
+
+The Kubernetes cluster on which you are deploying OpenWhisk must meet the following requirements:
+* [Kubernetes](https://github.com/kubernetes/kubernetes) version 1.9+. However, version 1.9.4
will not work for OpenWhisk due to a bug with volume mount subpaths (see[[kubernetes-61076](https://github.com/kubernetes/kubernetes/issues/61076)]).
This bug will surface as a failure when deploying the nginx container.
+* The ability to create Ingresses to make a Kubernetes service available outside of the cluster
so you can actually use OpenWhisk.
+* If you enable persistence (see [docs/configurationChoices.md](./docs/configurationChoices.md)),
either your cluster is configured to support [Dynamic Volume Provision](https://kubernetes.io/docs/concepts/storage/dynamic-provisioning/)
or you must manually create any necessary PersistentVolumes when deploying the Helm chart.
+* Endpoints of Kubernetes services must be able to loopback to themselves (the kubelet's
`hairpin-mode` must not be `none`).
+


Mime
View raw message