mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Janet Borschowa <janet.borsch...@codefutures.com>
Subject Re: Implementing an Executor
Date Thu, 20 Nov 2014 16:33:37 GMT
Hi Tom,
Thank you - this helps a lot!

Janet

On Wed, Nov 19, 2014 at 5:22 PM, Tom Arnfeld <tom@duedil.com> wrote:

> Hi Janet,
>
> Oh sorry my mistake, I didn't read your email correctly, I thought you
> were using the containerizer. What you're doing here is actually going to
> be quite difficult to do, the mesos docker containerizer has some quite
> complex logic implemented to ensure the slave stays in sync with the
> containers that are running, and kills anything that goes rogue.
>
> It's going to be non-trivial for you to do that from the executor, though
> I guess you could make use of the docker events API or poll other endpoints
> in the API to check the status of your containers, and off the back of that
> send status updates to the cluster. Doing this however brings no guarantees
> that if your executor dies exceptionally (perhaps OOMd) the containers
> spawned will die... they'll keep running in the background and it'll be
> hard for you to know the state of your containers on the cluster.
>
> You probably want to be aware (if you don't know already) that the
> resource limits assigned to your tasks aren't going to be enforced by mesos
> because docker is running outside of its control. You'll need to pass the
> correct CPU/Memory limit parameters to your docker containers to ensure
> this happens correctly.
>
> Here are the docker API docs;
> https://docs.docker.com/reference/api/docker_remote_api_v1.15/
>
> Something you might want to consider, if all you're trying to do is allow
> your container access to details about itself (e.g `docker inspect`) is to
> open up the docker remote API to be queried by your containers on the
> slave, and switch to using the mesos docker containerizer.
>
> I hope that helps somewhat!
>
> Tom.
>
> --
>
> Tom Arnfeld
> Developer // DueDil
>
> (+44) 7525940046
> 25 Christopher Street, London, EC2A 2BS
>
>
> On Wed, Nov 19, 2014 at 10:16 PM, Janet Borschowa <
> janet.borschowa@codefutures.com> wrote:
>
>>  Hi,
>> I'm implementing an executor which is used by the mesos slave to launch
>> tasks. The tasks are to launch a docker container - this is because I need
>> more info about the launched container than what the docker containerizer
>> returns.
>>
>> Is it OK to block in the executor's launchTask method until the task
>> completes? If not, how does the framework discover when that task
>> completes? I could spawn a process which notifies my executor when the task
>> completes and then have my executor send a status update. Or is there some
>> other recommended way to deal with this when the task could run for an
>> indefinite period of time before completing its work?
>>
>> Thanks!
>>
>> Janet
>>
>> --
>>  Janet Borschowa
>> CodeFutures Corporation
>>
>>
>>
>

Mime
View raw message