aurora-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joshua Cohen <jco...@apache.org>
Subject Re: Review Request 44685: Add scheduler support for running tasks using the mesos Docker containerizer.
Date Mon, 14 Mar 2016 16:16:26 GMT


> On March 14, 2016, 3:51 p.m., Joshua Cohen wrote:
> > Actually, one more thing... should we ensure that the task config it has either
executor config set or uses a Docker container? I think right now you could configure the
scheduler with `-require_docker_use_executor=False` and send a task with no processes and
no Docker container?
> 
> John Sirois wrote:
>     The existing code in this change should be enforcing this already; although at a
higher layer (the aurora config is checked for processes or Docker container, not the thrift
config derived from it (where the executor config is injected)).  Let me know if you see specific
misses in the code or unit test coverage.
> 
> John Sirois wrote:
>     ... and Bill's code in the scheduler checks the thrift config in this way upon receipt.

This is the scheduler change review where I was asking for this ;).

I see where we're ensuring that the command line flag is set properly if no executor is configured,
but I don't see where we ensure that if the flag is set then the incoming task has either
executor config or a docker container? It's possible I'm just missing it though!


- Joshua


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44685/#review123427
-----------------------------------------------------------


On March 11, 2016, 11:27 p.m., Bill Farner wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44685/
> -----------------------------------------------------------
> 
> (Updated March 11, 2016, 11:27 p.m.)
> 
> 
> Review request for Aurora, Joshua Cohen and John Sirois.
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> This is currently labeled as experimental.
> 
> Only the most basic wiring is added here, and assumes that the provided image
> includes an ENTRYPOINT.  Unlike Docker support via the thermos executor, this
> approach allows containers with an entrypoint, and does not impose environment
> requirements on the image (e.g. python interpreter, libmesos dependencies).
> 
> Note that when using this, other familiar Aurora facilities that relate to the
> thermos executor will not work.  For example, browsing task logs is not
> supported.
> 
> Support for exercising this from the client will come shortly.
> 
> 
> Diffs
> -----
> 
>   NEWS da3e4cea8ca688b6b7c5bafae67133df065d9255 
>   src/jmh/java/org/apache/aurora/benchmark/SnapshotBenchmarks.java 2c56b2eb91a942151865c5a2731ff1cea04b08eb

>   src/jmh/java/org/apache/aurora/benchmark/TaskStoreBenchmarks.java 2ec6abd04a454cdcf23f8480c3d14aea85bcb65d

>   src/jmh/java/org/apache/aurora/benchmark/ThriftApiBenchmarks.java 60746383fccb107ca27925a91aa1803e2cf0fd85

>   src/main/java/org/apache/aurora/scheduler/app/AppModule.java a0d2a717534bbb2e85a556721cc53c1e4b743461

>   src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 1de6966565d2fbd9abd220ad8162b624b109959a

>   src/main/java/org/apache/aurora/scheduler/configuration/ConfigurationManager.java e700fa3550312bfa9c8a3adb25d135f6f500c4b5

>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java a34af4d2fb3863ab8197bcdce942c513d629621b

>   src/main/java/org/apache/aurora/scheduler/storage/db/DbModule.java ff663fa6ea9c5f8b05dabf173ab8ee518fbdd055

>   src/main/java/org/apache/aurora/scheduler/storage/db/DbUtil.java 708be567ec0f8adbcc53be9553198a8cbb81f3b9

>   src/main/resources/org/apache/aurora/scheduler/storage/db/schema.sql c6c1f0a0f1ea64e27dd2b1aaf69ae5cd841c5657

>   src/test/java/org/apache/aurora/scheduler/app/local/FakeNonVolatileStorage.java 3336f8cddb2a7a6e8c9e4bc6664708342ab97979

>   src/test/java/org/apache/aurora/scheduler/configuration/ConfigurationManagerTest.java
11062e3a097e490c61bfd4dc84990903275521a3 
>   src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java 3db531b52fb2bd94b4b5ce62e6554b5a85ed3ea8

>   src/test/java/org/apache/aurora/scheduler/storage/db/DbCronJobStoreTest.java fa0d0dcd9975fd28b92e7a9ccddecbb9a01c01d8

>   src/test/java/org/apache/aurora/scheduler/storage/db/DbJobUpdateStoreTest.java 54defc256ad8a261c6b56ee06ad7fdd16a26b057

>   src/test/java/org/apache/aurora/scheduler/storage/db/DbTaskStoreTest.java ecddc66612751b53b71b917d05d0312aadc16076

>   src/test/java/org/apache/aurora/scheduler/storage/db/RowGarbageCollectorTest.java 58b4c9312e5c7d12144af56b3896161c039d5288

>   src/test/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImplIT.java 6a39d89bbba3430961f60a27547c3dfdeb9433f8

>   src/test/java/org/apache/aurora/scheduler/storage/mem/InMemTaskStoreTest.java d18ce205412efe08dcd41b6ebc79e29dde9320d1

>   src/test/java/org/apache/aurora/scheduler/storage/mem/MemCronJobStoreTest.java d3a026cedc86e366d140727dd98265804caed30b

>   src/test/java/org/apache/aurora/scheduler/thrift/ThriftIT.java 860d9607505c0a28c1d0050faa04143b0ab8a097

>   src/test/java/org/apache/aurora/scheduler/updater/JobUpdaterIT.java cc8891593c6f31356986f3eb66aabd04ae97acbb

> 
> Diff: https://reviews.apache.org/r/44685/diff/
> 
> 
> Testing
> -------
> 
> Via additional hacking, i successfully ran the stock [hello world](https://hub.docker.com/_/hello-world/)
image.  Within the sandbox, i observed the expected output in the `stdout` file.  Status updates
for the task exiting worked as expected.
> 
> 
> Thanks,
> 
> Bill Farner
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message