incubator-mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Challoner <>
Subject Best way to go about writing a framework for daemon-like tasks
Date Sat, 20 Apr 2013 19:00:37 GMT
Hi, new to mesos.   I setup a test cluster in EC2 (which required some
tweaks to the provided scripts - i'll try to send those back in) but I'm
not sure how I should write the framework for what I'd like to achieve.

I'd like to use Mesos to run a dynamically changing list of applications
like so:
{ job: app,
  cpu: 3,
  memory: 20g,
  instances: 4},
 { job: app2,
  cpu: 5,
  memory: 30g,
  instances: 2}

So I want the framework to pull the list out of a database/redis/zk and run
these apps across the cluster in a round robbin fashion until either the
cluster resources are exhausted or we've satisfied the number of running
instances.   What I'm having trouble groking is how this would look on a
framework/executor level.

With a single framework it seems you're likely to fail using all available
resources given a long running daemon-like tasks:
*framework registered
*node offers 10 cpus
*framework accepts offer and at the time decides to to gives it 8cpu worth
of tasks to run.
*node has 2 cpus left over
*at some point the list of apps changes and a new app allocation is
needed..  Maybe we have a new app that could use those 2 cpus or maybe we
just need to adjust how many of the old apps are running.  If I understand
the docs correctly, the nodes won't re-offer because they've already been
assigned tasks by the framework that will run forever.

Do I maybe submit a new framework for each app type then?  Would that scale
to large numbers of apps?

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message