aurora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Renan DelValle <rdelv...@binghamton.edu>
Subject Settling on a custom executors direction
Date Thu, 06 Aug 2015 00:48:05 GMT
Hi all,

I wanted to discuss with the community the direction custom executor
support will take in Aurora. I think it's a good time to start a
discussion on the vision everyone has in order to determine the best
direction to take.

I hope we can settle on something concrete vision sooner than later in
order to deliver this feature in a timely manner (hopefully before
Mesoscon!).

Summary of what's been discussed so far:

What's been settled:

* Configuration for executor should be be loaded by the scheduler for
security reasons. (At least for the time being)

 (https://issues.apache.org/jira/browse/AURORA-1288?focusedCommentId=14601470&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14601470)

* It would be better to create this feature in such a way that it's
prepared to adapt the future REST API. Therefore, it doesn't make
sense to extend the DSL to support custom executors. The consequence
of this is that a custom client will be necessary to run custom
executors (for now).

(http://mail-archives.apache.org/mod_mbox/aurora-dev/201507.mbox/%3CCAGRA8uNPsaVJud1HObGFfq9jN43TxJK3CUe-SzfWwP8by_gn%2Bg%40mail.gmail.com%3E)

*  We will be using a config file (as discussed earlier in the mailing
list) and it will be in JSON

Things up in the air:

* What should the config file look like? E.g.: Should it mimic
ExecutorSettings? Should it mimic the Mesos TaskInfo? If we mimic
TaskInfo, do we ditch ExecutorSettings?

* Do we provide an interface to allow users to build the Mesos tasks
themselves? Or do we have a generic Mesos task with some required
fields and some not required?

 - If we take the route of using a generic Mesos tasks, how do we take
this direction with minimal effect on thermos specific settings?

  - If we take the route of allowing users to build the tasks
themselves,  what would be the best way to accomplish this? For
example, how should we call the correct task builder? Do we make it
dynamic based on the executor name per job or static for the entire
scheduler?

  - Some discussion about this has been taking place here
(https://reviews.apache.org/r/36289/)

Looking forward to hearing some thoughts on this.

-Renan

Mime
View raw message