mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kone <vinodk...@gmail.com>
Subject Re: Porting an app
Date Fri, 27 Dec 2013 18:10:58 GMT
>
> I can't really find an example that is an end-to-end use case. By that I
> mean, I would like to know how to put the scheduler and the executor in the
> correct places. Right now I have a single jar with can be run from the
> command line: java -jar target/collector.jar and that would take care of
> everything.
>
> This collector.jar can act as both scheduler and executor, presumably
based on command line flags? If yes, thats certainly doable. Typically the
scheduler and executor are split into separate jars. This makes it easy to
decouple the upgrade of scheduler and executor.


> My current train of thought is that the webapp jar would stay somewhere on
> an S3 url and the "CollectorScheduler" would "somehow" tell a mesos slave
> to run the "CollectorExecutor" which in turn fetch the jar from S3 and run
> it.
>
>
Yes, you are on the right track. Mesos slave can download the jar for you
as long as it could be accessed via (http://, https://, ftp://, hdfs://
etc). This is how you do it:

When you launch a task from the scheduler via 'launchTaks()' you give it a
'vector<TaskInfo>' as one of the arguments. Since you are using a custom
executor you should set 'TaskInfo.ExecutorInfo' (see mesos.proto) to point
to your executor. To specify the S3 URL you would set
'TaskInfo.ExecutorInfo.CommandInfo.URI.value'. To tell slave the command to
launch the executor after it downloads the the executor, you would set
'TaskInfo.ExecutorInfo.CommandInfo.value'.

You can find some examples here:

Hadoop scheduler<https://github.com/mesos/hadoop/blob/master/src/main/java/org/apache/hadoop/mapred/ResourcePolicy.java>

Example Java scheduler
<https://github.com/apache/mesos/blob/master/src/examples/java/TestFramework.java>

Hope that helps. Let us know if you have additional questions.

Vinod

Mime
View raw message