mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Pampuch <j...@mesosphere.io>
Subject Re: multiple frameworks or one big one
Date Thu, 26 Feb 2015 02:02:22 GMT
Diego-

You might want to look at this project for some insights:

https://github.com/mesosphere/RENDLER


-John


On Wed, Feb 25, 2015 at 5:27 PM, Diego Medina <diego@fmpwizard.com> wrote:

> Hi,
>
>
> Short: Is it better to have one big framework and executor with if
> statements to select what to do or several smaller framework <-> executors
> when writing a Mesos app?
>
> Longer question:
>
> Last week I started a side project based on mesos (using Go),
>
> http://blog.fmpwizard.com/blog/web-crawler-using-mesos-and-golang
> https://github.com/fmpwizard/owlcrawler
>
> It's a web crawler written on top of Mesos, The very first version of it
> had a framework that sent a task to an executor and that single executor
> would fetch the page, extract links from the html and then send them to a
> message queue.
>
> Then the framework reads that queue and starts again, run the executor,
> etc, etc.
>
> Now I'm splitting fetching the html and extracting links into two separate
> tasks, and putting those two tasks in the same executor doesn't feel right,
> so I'm thinking that I need at least two diff executors and one framework,
> but then I wonder if people more experienced with mesos would normally
> write several pairs of framework <-> executors to keep the design cleaner.
>
> On this particular case, I can see the project growing into even more
> tasks that can be decoupled.
>
> Any feedback on the design would be great and let me know if I should
> explain this better.
>
> Thanks
>
>
> Diego
>
>
>
>
>
> --
> Diego Medina
> Lift/Scala consultant
> diego@fmpwizard.com
> http://fmpwizard.telegr.am
>

Mime
View raw message