mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dominic Hamon <dha...@twitter.com.INVALID>
Subject Re: Mesos Modules Design
Date Mon, 22 Sep 2014 22:09:13 GMT
On Mon, Sep 22, 2014 at 2:36 PM, Benjamin Hindman <benh@eecs.berkeley.edu>
wrote:

> Jumping on a tcon/hangout sounds healthy, but given I'm traveling right now
> in Europe and timing is difficult I'm going to comment inline here.
>
>
> - callsites need to be modules aware to use the right factory method to
> > instantiate the modular object
> >
>
> I don't know how else you'd accomplish making certain components of the
> system *modular* unless we changed the way we instantiated the components.
> This one seems fairly self-evident to me, but perhaps there's a trick we
> could play instead? Are you thinking something LD_PRELOAD-esque here? But
> works well with C++?
>

​I'm thinking that you'd instantiate a module using the same class name but
the link step would take care of making sure the modular implementation is
available.​



>
>
> > - mesos-core owners need to be aware of modules and versioning when they
> > change the abstract API
> >
>
> Yes, definitely, anyone hacking in Mesos should be aware that changing the
> API of something like the Allocator might break all Allocator
> implementations that exist out there. This is pretty fundamental to module
> systems and one of the reasons why the design has a flexible, albeit more
> complicated, verification/versioning component.
>
>
> > - module owners need to be aware of the modules API and versioning
> >
>
> As they should as well, since it would be nice to give us some flexibility
> in the future to deal with the mistakes we're bound to make.
>
>
> > - customers (end-users) have to be modules aware and set their runtime
> > configuration correctly
> >
>
> Mesos will always function without the need for any external modules, as it
> does today. But yes, if an operator wants to use a module than they will
> definitely need to update the configuration to do so. The example I'm
> thinking about here is LoadModule from Apache HTTPD.
>
>
> > - errors in versioning or modules will only be caught at runtime
> >
>
> This is an artifact of the design constraints. In fact, perhaps it wasn't
> highlighted explicitly in the design document, but we'd definitely like to
> support modules in binary format, like an RPM (as you can do with Apache
> HTTPD modules as well). That implies that a module developer might be
> building their module independently of an operator building Mesos.
> Moreover, it implies that if there is anything we definitely want to make
> rock solid in our design it's what happens when such a module is launched
> with a running Mesos cluster, hence the design's focus on verification and
> versioning. Perhaps the design is too complicated and we should settle on
> something simpler, like forcing every build of a module to be associated
> with a single Mesos version, but regardless, an operator might still make a
> mistake when configuring Mesos and we must do the verification at runtime.
>


​Unless you take the configuration of modules out of runtime and into
compile or link time.​



>
>
> > My alternative proposal requires the following:
> >
>
> Sorry if I'm just out of the loop here, but did you share a proposal with
> the mailing list or on JIRA that I missed? I'm having a hard time following
> your statements about your proposal without that context. That would
> definitely help facilitate a discussion via tcon/hangout as well.
>


​I've made this proposal on JIRA and on this mailing list, but for
simplicity here it is again:

- create abstract classes to define interfaces to objects that should be
modular
- build modules as static libraries that can be assembled at link time to
create custom Mesos builds

That's it. :)​




-- 
Dominic Hamon | @mrdo | Twitter
*There are no bad ideas; only good ideas that go horribly wrong.*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message