airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [airflow] gerardo commented on a change in pull request #5837: [AIRFLOW-5223] Use KinD for Kubernetes in CI
Date Wed, 04 Sep 2019 01:06:57 GMT
gerardo commented on a change in pull request #5837: [AIRFLOW-5223] Use KinD for Kubernetes
in CI
URL: https://github.com/apache/airflow/pull/5837#discussion_r320536584
 
 

 ##########
 File path: scripts/ci/kubernetes/docker/build.sh
 ##########
 @@ -20,29 +20,14 @@ IMAGE=${IMAGE:-airflow}
 TAG=${TAG:-latest}
 DIRNAME=$(cd "$(dirname "$0")" && pwd)
 AIRFLOW_ROOT="${DIRNAME}/../../../.."
-PYTHON_DOCKER_IMAGE=python:3.6-slim
 
 set -e
 
-# Don't rebuild the image more than once on travis
-if [[ -n "${TRAVIS}" || -z "${AIRFLOW_CI_REUSE_K8S_IMAGE}" ]] && \
-    docker image inspect "${IMAGE}:${TAG}" > /dev/null 2>/dev/null; then
-  echo "Re-using existing image"
-  exit 0
-fi
-
-if [[ "${VM_DRIVER:-none}" != "none" ]]; then
-    if ENVCONFIG=$(minikube docker-env); then
-      eval "${ENVCONFIG}"
-    fi
-fi
-
 echo "Airflow directory ${AIRFLOW_ROOT}"
 echo "Airflow Docker directory ${DIRNAME}"
 
 cd "${AIRFLOW_ROOT}"
-docker run -ti --rm -v "${AIRFLOW_ROOT}:/airflow" \
-    -w /airflow "${PYTHON_DOCKER_IMAGE}" ./scripts/ci/kubernetes/docker/compile.sh
+python setup.py compile_assets sdist -q
 
 Review comment:
   Actually, this implementation runs docker-in-docker, so we actually need to do `service
docker start` because the daemon runs _inside_ the `airflow-testing` container. Defining `/var/lib/docker`
as a volume is still needed, but now that you mention it, I'm not completely sure we still
need `priviledged:true`. I'll verify.
   
   The setup you describe is where we mount `/var/lib/docker.sock`, the host's Docker socket
in the container. I actually implemented that solution first. It works, but I thought that
a more self-contained solution would be a better fit for a developer workflow where people
just clone the repo and get to work and not have to deal with mounting sockets from the host.
 I've had problems in the past, especially in Linux, where you had to fix `/var/lib/docker.sock`s
permissions or relax SELinux's policies.
   
   But for the most part, I agree with you, I'd prefer to reuse the images already available.
Of course, if they have everything that the Kubernetes tests expect, which I haven't checked
yet.
   
   With all this info, what's your opinion then on mounting `/var/lib/docker.sock` vs having
a fully self-contained solution?
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message