mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim St Clair <tstcl...@redhat.com>
Subject Re: Mesos Modules Design
Date Mon, 22 Sep 2014 14:15:05 GMT
Folks, 

I think this thread has fallen off the rails a bit, perhaps we could get a tcon/hangout going
for this again.  

I would be happy to moderate if needed, but it's pretty clear to me to that trying to hash
this out in an email thread is becoming couter-productive.

Cheers,
Tim

----- Original Message -----
> From: "Dominic Hamon" <dhamon@twitter.com.INVALID>
> To: "dev" <dev@mesos.apache.org>
> Sent: Saturday, September 20, 2014 9:21:59 AM
> Subject: Re: Mesos Modules Design
> 
> On Fri, Sep 19, 2014 at 2:43 PM, Niklas Nielsen <niklas@mesosphere.io>
> wrote:
> 
> > Hi Dominic,
> >
> > (response inlined)
> >
> > On 19 September 2014 13:03, Dominic Hamon <dhamon@twopensource.com> wrote:
> >
> > > I'm sorry, but I'm still having a hard time understanding why this needs
> > to
> > > be dynamic.
> > >
> > > If the mesos core is split into modules that are built as standalone
> > > libraries (static) then at link time the right combination of libraries
> > can
> > > be bundled together to create the end result. If you want to get even
> > > smarter, we can have default versions that are linked in to the mesos
> > core
> > > as weak symbols so later linked libraries can override the defaults. This
> > > may mean that we move to static linking across the board, but frankly
> > there
> > > are a few benefits to that approach.
> >
> >
> > This works for some, but not all use cases.
> > One use-case where it _does_ make sense to statically bake into the image
> > could be: https://issues.apache.org/jira/browse/MESOS-1330
> > That be, you probably rarely want to swap network transport implementations
> > in and out on per-run basis but will know up front which one to configure
> > and use.
> >
> > On the other hand, having to relink and rebuilding give users a poor
> > experience and makes it hard to select (or unselect) custom components.
> > Using a prebuilt package and point against libraries is a pretty common
> > work-flow: Apache web server relies heavily on modules from dynamic
> > libraries.
> >
> >
> ‚ÄčI'm going to argue against this, so please bear with me while I work
> through it, and please point out anywhere that my assertions are wrong.
> 
> The proposal you are suggesting requires the following:
> 
> - callsites need to be modules aware to use the right factory method to
> instantiate the modular object
> - mesos-core owners need to be aware of modules and versioning when they
> change the abstract API
> - module owners need to be aware of the modules API and versioning
> - customers (end-users) have to be modules aware and set their runtime
> configuration correctly
> - errors in versioning or modules will only be caught at runtime
> 
> My alternative proposal requires the following:
> 
> -  module owners need to be aware of API changes during core upgrades
> - issues are caught at compile time by module owners
> - customers (end-users) are unaware of modules and only need to know about
> per module configuration (kerberos attributes, for instance) not the
> modules system and configuration
> 
> In short, your proposal adds more complexity and shifts the burden of
> knowledge and responsibility to every agent, instead of keeping it with the
> module owner where it belongs.
> 
> I think the solution that's been put up for review is clever, but I don't
> think it solves the problem in the right way for anyone involved. Please
> rethink your approach to take that into account.
> 
> 
> 
> > TL;DR I don't see the two approaches to be mutually exclusive and we can
> > get a lot of leverage with the current design but we want to be able to do
> > this with static linking too. (To Tim St Clair's point)
> >
> >
> > >
> >
> >
> >
> > > With the approach as defined, does this mean that the default versions
> > will
> > > also have to be reimplemented as modules?
> > >
> >
> > Not necessary. Mesos default implementations stays as is. See Bernd's
> > comment.
> >
> >
> > >
> > > Has any effort been put into determining the performance overhead of the
> > > approach as specified?
> > >
> >
> > It won't affect Mesos if you are not using modules. The call sites will be
> > virtual dispatches no matter whether you are using modules or
> > internal/default implementations.
> > There is a performance penalty of not being able to do global optimizations
> > within the module, but that is the trade-off of implementing a dynamic
> > loadable module.
> >
> >
> > >
> > > On Fri, Sep 19, 2014 at 11:35 AM, Niklas Nielsen <niklas@mesosphere.io>
> > > wrote:
> > >
> > > > Hi everyone,
> > > >
> > > > We have been iterating on a design for pluggable modules in Mesos
> > lately
> > > > and wanted to get a last round of feedback before putting out patch
> > sets.
> > > >
> > > > Tim St Clair, Ben Hindman and I started the discussion (and work) on
> > this
> > > > subsystem https://issues.apache.org/jira/browse/MESOS-1224 and
> > > > https://issues.apache.org/jira/browse/MESOS-1384.
> > > > Kapil and Bernd took over the work (shepherded by Ben H and I) and have
> > > > expanded on the original design to cope with api/modules/mesos
> > versioning
> > > > semantics and be extensible enough to cope future changes in the
> > modules
> > > > subsystem (dealing with modules dependencies, etc).
> > > >
> > > > The latest description of the modules system has been captured in:
> > > >
> > > >
> > >
> > https://cwiki.apache.org/confluence/display/MESOS/DRAFT+Design+Doc+-+Mesos+Modules
> > > > (for those of you who don't want to read through the JIRA threads).
> > > >
> > > > We have an implementation ready based on this design and will be
> > sharing
> > > /
> > > > starting review rounds start next week.
> > > >
> > > > Feel free to use this thread if you have any questions.
> > > >
> > > > Cheers,
> > > > Niklas
> > > >
> > >
> > >
> > >
> > > --
> > > Dominic Hamon | @mrdo | Twitter
> > > *There are no bad ideas; only good ideas that go horribly wrong.*
> > >
> >
> > Hope that helps; if not, we'd be happy to jump on a call.
> >
> > Niklas
> >
> 
> 
> 
> --
> Dominic Hamon | @mrdo | Twitter
> *There are no bad ideas; only good ideas that go horribly wrong.*
> 

-- 
Cheers,
Timothy St. Clair
Red Hat Inc.

Mime
View raw message