yetus-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From a.@apache.org
Subject yetus git commit: YETUS-561. Ability to limit user process counts and Docker container's RAM usage
Date Sat, 28 Oct 2017 12:32:11 GMT
Repository: yetus
Updated Branches:
  refs/heads/master 959d46c60 -> d8795e0fb


YETUS-561. Ability to limit user process counts and Docker container's RAM usage

Signed-off-by: Sean Busbey <busbey@apache.org>


Project: http://git-wip-us.apache.org/repos/asf/yetus/repo
Commit: http://git-wip-us.apache.org/repos/asf/yetus/commit/d8795e0f
Tree: http://git-wip-us.apache.org/repos/asf/yetus/tree/d8795e0f
Diff: http://git-wip-us.apache.org/repos/asf/yetus/diff/d8795e0f

Branch: refs/heads/master
Commit: d8795e0fbdd97b89ec31c97904e79ed6cffb4426
Parents: 959d46c
Author: Allen Wittenauer <aw@apache.org>
Authored: Tue Oct 24 09:46:01 2017 -0700
Committer: Allen Wittenauer <aw@apache.org>
Committed: Fri Oct 27 21:37:20 2017 -0700

----------------------------------------------------------------------
 .../in-progress/precommit-advanced.md              |  4 ++++
 .../documentation/in-progress/precommit-basic.md   | 10 ++++++++++
 precommit/core.d/docker.sh                         | 17 +++++++++++++++--
 precommit/test-patch.sh                            |  9 +++++++++
 4 files changed, 38 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/yetus/blob/d8795e0f/asf-site-src/source/documentation/in-progress/precommit-advanced.md
----------------------------------------------------------------------
diff --git a/asf-site-src/source/documentation/in-progress/precommit-advanced.md b/asf-site-src/source/documentation/in-progress/precommit-advanced.md
index 264f273..159e154 100644
--- a/asf-site-src/source/documentation/in-progress/precommit-advanced.md
+++ b/asf-site-src/source/documentation/in-progress/precommit-advanced.md
@@ -47,6 +47,10 @@ NOTE: If you are using Boot2Docker, you must use directories under /Users
(OSX)
 
 Dockerfile images will be named with a test-patch prefix and suffix with either a date or
a git commit hash. By using this information, test-patch will automatically manage broken/stale
container images that are hanging around if it is run in --robot mode.  In this way, if Docker
fails to build the image, the disk space should eventually be cleaned and returned back to
the system.  The docker mode can also be run in a "safe" mode that prevents deletions via
the `--dockerdelrep` option.  Specifying this option will cause test-patch to only report
what it would have deleted, but not actually remove anything.
 
+Docker's `--memory` flag is supported via the `--dockermemlimit` option.  This enables the
container's memory size to be limited.  This may be important to set to prevent things like
broken unit tests bringing down the entire build server.  See  [the Docker documentation](https://docs.docker.com/engine/admin/resource_constraints/)
for more details.
+
+Additionally, Apache Yetus sets the --oom-score-adj to 500 in order to offer itself as the
first processes to be killed if memory is low.
+
 # Plug-ins
 
 test-patch allows one to add to its basic feature set via plug-ins.  There is a directory
called test-patch.d inside the directory where test-patch.sh lives.  Inside this directory
one may place some bash shell fragments that, if setup with proper functions, will allow for
test-patch to call it as necessary.  Different plug-ins have specific functions for that particular
functionality.  In this document, the common functions available to all/most plug-ins are
covered.  Test plugins are covered below. See other documentation for pertinent information
for the other plug-in types.

http://git-wip-us.apache.org/repos/asf/yetus/blob/d8795e0f/asf-site-src/source/documentation/in-progress/precommit-basic.md
----------------------------------------------------------------------
diff --git a/asf-site-src/source/documentation/in-progress/precommit-basic.md b/asf-site-src/source/documentation/in-progress/precommit-basic.md
index b6b99aa..1a82beb 100644
--- a/asf-site-src/source/documentation/in-progress/precommit-basic.md
+++ b/asf-site-src/source/documentation/in-progress/precommit-basic.md
@@ -162,6 +162,16 @@ $ test-patch.sh --basedir=<testrepo> --resetrepo /tmp/patchfile
 
 We used two new options here.  --basedir sets the location of the repository to use for testing.
 --resetrepo tells test patch that it can go into **destructive** mode.  Destructive mode
will wipe out any changes made to that repository, so use it with care!
 
+# Fork Bomb Protection
+
+By default, test-patch.sh will set the user soft limit (ulimit -Su) to a relatively low 1,000
processes (and, on some operating systems, threads!). This is to prevent errant processes
from eating up all system resources.  If this limit is too low (e.g., highly threaded Java
processes), it may be necessary to use the `--proclimit` option.  For example:
+
+```bash
+$ test-patch --proclimit=10000
+```
+
+... will set it to be 10,000 processes.
+
 # Automation
 
 After the tests have run, there is a directory that contains all of the test-patch related
artifacts.  This is generally referred to as the patchprocess directory.  By default, test-patch
tries to make something off of /tmp to contain this content.  Using the `--patch-dir` option,
one can specify exactly which directory to use.  This is helpful for automated precommit testing
so that Jenkins or other automated workflow system knows where to look to gather up the output.

http://git-wip-us.apache.org/repos/asf/yetus/blob/d8795e0f/precommit/core.d/docker.sh
----------------------------------------------------------------------
diff --git a/precommit/core.d/docker.sh b/precommit/core.d/docker.sh
index 20a9ade..48cba0a 100755
--- a/precommit/core.d/docker.sh
+++ b/precommit/core.d/docker.sh
@@ -23,6 +23,7 @@ DOCKERFAIL="fallback,continue,fail"
 DOCKERSUPPORT=false
 DOCKER_ENABLE_PRIVILEGED=true
 DOCKER_CLEANUP_CMD=false
+DOCKER_MEMORY="4g"
 
 declare -a DOCKER_EXTRAARGS
 
@@ -54,6 +55,8 @@ function docker_usage
     yetus_add_option "--dockerprivd=<bool>" "Run docker in privileged mode (default:
'${DOCKER_ENABLE_PRIVILEGED}')"
   fi
   yetus_add_option "--dockerdelrep" "In Docker mode, only report image/container deletions,
not act on them"
+  yetus_add_option "--dockermemlimit=<num>" "Limit a Docker container's memory usage
(default: ${DOCKER_MEMORY})"
+
 }
 
 ## @description  Docker-specific argument parsing
@@ -80,6 +83,9 @@ function docker_parse_args
       --dockerfile=*)
         DOCKERFILE=${i#*=}
       ;;
+      --dockermemlimit=*)
+        DOCKER_MEMORY=${i#*=}
+      ;;
       --dockermode)
         DOCKERMODE=true
       ;;
@@ -601,14 +607,21 @@ PatchSpecificDocker
   fi
 
   if [[ "${DOCKER_ENABLE_PRIVILEGED}" = true ]]; then
-    DOCKER_EXTRAARGS=("--privileged" "${DOCKER_EXTRAARGS[@]}")
+    DOCKER_EXTRAARGS+=("--privileged")
   fi
 
   if [[ -n "${CONSOLE_REPORT_FILE}" ]]; then
     touch "${CONSOLE_REPORT_FILE}"
-    DOCKER_EXTRAARGS=("${DOCKER_EXTRAARGS[@]}" "-v" "${CONSOLE_REPORT_FILE}:/testptch/console.txt")
+    DOCKER_EXTRAARGS+=("-v" "${CONSOLE_REPORT_FILE}:/testptch/console.txt")
   fi
 
+  if [[ -n "${DOCKER_MEMORY}" ]]; then
+    DOCKER_EXTRAARGS+=("-m" "${DOCKER_MEMORY}")
+  fi
+
+  # make the kernel prefer to kill us if we run out of RAM
+  DOCKER_EXTRAARGS+=("--oom-score-adj" "500")
+
   client=$(docker_version Client)
   server=$(docker_version Server)
 

http://git-wip-us.apache.org/repos/asf/yetus/blob/d8795e0f/precommit/test-patch.sh
----------------------------------------------------------------------
diff --git a/precommit/test-patch.sh b/precommit/test-patch.sh
index ccc9990..8b1a8cc 100755
--- a/precommit/test-patch.sh
+++ b/precommit/test-patch.sh
@@ -91,6 +91,7 @@ function setup_defaults
 
   # shellcheck disable=SC2034
   CHANGED_UNION_MODULES=""
+  PROC_LIMIT=1000
   REEXECED=false
   RESETREPO=false
   BUILDMODE=patch
@@ -711,6 +712,7 @@ function yetus_usage
   yetus_add_option "--offline" "Avoid connecting to the Internet"
   yetus_add_option "--patch-dir=<dir>" "The directory for working and output files
(default '/tmp/test-patch-${PROJECT_NAME}/pid')"
   yetus_add_option "--personality=<file>" "The personality file to load"
+  yetus_add_option "--proclimit=<num>" "Limit on the number of processes (default:
${PROC_LIMIT})"
   yetus_add_option "--project=<name>" "The short name for project currently using test-patch
(default 'yetus')"
   yetus_add_option "--plugins=<list>" "Specify which plug-ins to add/delete (comma
delimited; use 'all' for all found) e.g. --plugins=all,-ant,-scalac (all plugins except ant
and scalac)"
   yetus_add_option "--resetrepo" "Forcibly clean the repo"
@@ -867,6 +869,9 @@ function parse_args
       --personality=*)
         PERSONALITY=${i#*=}
       ;;
+      --proclimit=*)
+        PROC_LIMIT=${i#*=}
+      ;;
       --reexec)
         REEXECED=true
       ;;
@@ -3122,6 +3127,10 @@ else
   initialize "$@"
 fi
 
+ulimit -Su "${PROC_LIMIT}"
+
+yetus_debug "Changed process/Java native thread limit to ${PROC_LIMIT}"
+
 add_vote_table H "Prechecks"
 
 prechecks


Mime
View raw message