aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joshua Cohen <jco...@apache.org>
Subject Review Request 47853: Isolate the executor's filesystem from the task's.
Date Wed, 25 May 2016 21:18:40 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47853/
-----------------------------------------------------------

Review request for Aurora, Maxim Khutornenko and Stephan Erb.


Repository: aurora


Description
-------

This changes the approach to launching tasks with filesystem images in the unified containerizer.
Instead of adding an `Image` to the `MesosContainer`, we instead add the task filesystem as
a `Volume` with an associated image. This image is mounted in the mesos directory under the
`taskfs` path. The executor, on start up does the following:

1. Creates user/group under the taskfs root.
2. `pivot_root`s into the taskfs, while bind mounting the sandbox under that root as well
as mounting procfs.
3. From there, task execution is essentially unchanged minus some slight changes to the environment
depending on whether we're running in a pivoted root.


Diffs
-----

  api/src/main/thrift/org/apache/aurora/gen/api.thrift a99889c1f2d9e10825f87ea669532ad78641880f

  examples/vagrant/upstart/aurora-scheduler.conf 3d9e706de564df5e24cb34265bebc0db1cad11a0

  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 3b01801d929dd61ee989495bf38af8f03e9f5ad4

  src/main/python/apache/aurora/executor/common/sandbox.py be1deba6219462c9fdaaf07a583851b85fe974bf

  src/main/python/apache/aurora/executor/thermos_task_runner.py 3896e3841562600379705dbf78a6f62728246348

  src/main/python/apache/thermos/core/BUILD 1094664e112cc71af37835f32037e9eb6d047202 
  src/main/python/apache/thermos/core/process.py 1791b5ff9a36eef7470bef9a6ebbafaf0ab05ca3

  src/main/python/apache/thermos/core/runner.py 3ebf86ebd12ed3b68f543d4b9a45615e4681ba7f 
  src/main/python/apache/thermos/runner/thermos_runner.py 0d06e8e2ac78d26ba8f63744853eb5ce3f6aced6

  src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 58785bfa37ff214f26e9f94d836e6df40e411c3b

  src/test/python/apache/aurora/executor/common/test_sandbox.py e47d9b8822deb36cb9cfa0554ef89d6cda80f3e9

  src/test/python/apache/thermos/core/test_process.py 77f644c09116266ce02479b9a80403aa68767bd6

  src/test/sh/org/apache/aurora/e2e/Dockerfile 6fdea3d28760f59235c51c5b6913d2ee0172ef1a 
  src/test/sh/org/apache/aurora/e2e/Dockerfile.netcat PRE-CREATION 
  src/test/sh/org/apache/aurora/e2e/http/http_example.aurora 219c40fb94561f0a390cac16e643bf4332c51aad

  src/test/sh/org/apache/aurora/e2e/http/http_example_bad_healthcheck.aurora 08553e4f48f137e0455ad07287086311171c06bd

  src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora 8b3a50ba6de992560593987f3db254baa4d29a41

  src/test/sh/org/apache/aurora/e2e/run-server.sh PRE-CREATION 
  src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh abe0ca75c6a2c0ace15fce68ad0e5c9aa98193a4


Diff: https://reviews.apache.org/r/47853/diff/


Testing
-------

Lots of manual testing, e2e tests, etc.

I didn't add much coverage on the thermos side of things because it seemed like this was better
served by the e2e tests than by doing a bunch of subprocess.check_call mocking. On the e2e
front I created a new Dockerfile that sets up a much slimmer filesystem image that explicitly
does not include python to ensure that the executor's filesystem is truly isolated.


Thanks,

Joshua Cohen


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message