cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Darren Shepherd <darren.s.sheph...@gmail.com>
Subject Re: [PROPOSAL] Modularize Spring
Date Thu, 19 Sep 2013 02:06:03 GMT
Yes this is one of the many things this is trying to address.  Adding a
plugin should be plopping your jar in a directory and restarting.  You
pointed out two things I didn't think about though, command.properties and
log4j xml.  Let me think about those twos as they should be address also.
Basically you should never have to edit a file that is packaged as part of
ACS.  Only add your artifacts to some directory, ideally just a jar.

Darren


On Wed, Sep 18, 2013 at 5:46 PM, SuichII, Christopher <
Chris.Suich@netapp.com> wrote:

> I've been following this conversation somewhat and would like to throw in
> some background as a plugin writer.
>
> One thing that concerns me in the current plugin model is the number of
> XML/text files that need to be edited to deploy my plugin.
> -applicationContext must be edited to add our PluginMangerImpl.
> -commands.properties file must be edited to included the permissions for
> the APIs we contributed.
> -componentContext.xml & nonossComponentContext.xml must be edited to add
> our Storage Subsystem Provider API.
> -log4j-cloud.xml must be edited to ensure that our logger is enabled and
> logging to our necessary default level.
>
> I know our situation is a bit different than the current plug in model,
> but I think it is something we, as a community, are going to begin seeing
> more of. For a variety of reasons that I won't get in to right now, our
> plugin will be closed source and kept separate from the ACS source tree. We
> want our users to be able to simply drop in our jar file to the CS
> directory or run and installer and have it picked up by the MS upon a
> restart.
>
> It sounds like what you are proposing here would be very beneficial to
> plugins that are targeting a deployment model like this.
>
> Is this something you're looking/hoping/would like to solve, Darren?
>
> -Chris
> --
> Chris Suich
> chris.suich@netapp.com
> NetApp Software Engineer
> Data Center Platforms – Cloud Solutions
> Citrix, Cisco & Red Hat
>
> On Sep 18, 2013, at 6:44 PM, Darren Shepherd <darren.s.shepherd@gmail.com>
> wrote:
>
> > I'm not a committer
> >
> >
> > On Wed, Sep 18, 2013 at 3:24 PM, Frank Zhang <Frank.Zhang@citrix.com>
> wrote:
> >
> >> Well. The codes explain more than words.
> >> It seems the only extra work is adding a property file that specifies
> >> parent context and current context name, it's not much complex.
> >> BTW: any reason for working on repo outside ACS?
> >>
> >>> -----Original Message-----
> >>> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
> >>> Sent: Wednesday, September 18, 2013 2:43 PM
> >>> To: dev@cloudstack.apache.org
> >>> Subject: Re: [PROPOSAL] Modularize Spring
> >>>
> >>> If you want to see this all working you can just fetch the "no-at-db4"
> >>> branch at https://github.com/ibuildthecloud/cloudstack.git
> >>>
> >>> Plugin composes multiple modules.  If modules are siblings they can't
> >> inject
> >>> from each other.  But a plugin can augment another module if it chooses
> >> to.
> >>> The reality is that the core cloudstack is a tangled mess of
> >> dependencies such
> >>> that most of the core code can't be modularized as it stands.  So there
> >> exists a
> >>> context towards the top of the hierarchy called "core" that a lot of
> jars
> >>> contribute to it.  Here is the full hierarchy right now.  I'll probably
> >> rename a
> >>> bunch of things, but this gives you an idea.
> >>>
> >>> bootstrap
> >>>  system
> >>>    core
> >>>      allocator
> >>>        allocator-server
> >>>        planner
> >>>          api-planner
> >>>          baremetal-planner
> >>>          explicit-dedication
> >>>          host-anti-affinity
> >>>          implicit-dedication-planner
> >>>          server-planner
> >>>          user-concentrated-pod-planner
> >>>        random-allocator
> >>>      api
> >>>        acl-static-role-based
> >>>        rate-limit
> >>>        server-api
> >>>        user-authenticator-ldap
> >>>        user-authenticator-md5
> >>>        user-authenticator-plaintext
> >>>        user-authenticator-sha256salted
> >>>      backend
> >>>        alert-adapter-server-backend
> >>>        compute
> >>>          alert-adapter-server-compute
> >>>          baremetal-compute
> >>>          fencer-server
> >>>          investigator-server
> >>>          kvm-compute
> >>>          ovm-compute
> >>>          server-compute
> >>>          xenserver-compute
> >>>        network
> >>>          baremetal-network
> >>>          elb
> >>>          midonet
> >>>          nvp
> >>>          ovs
> >>>          server-network
> >>>          ssp
> >>>        storage
> >>>          alert-adapter-server-storage
> >>>          allocator-storage
> >>>          baremetal-storage
> >>>          secondary-storage
> >>>          server-storage
> >>>          storage-image-default
> >>>          storage-image-s3
> >>>          storage-image-swift
> >>>          storage-volume-default
> >>>          storage-volume-solidfire
> >>>          template-adapter-server
> >>>      discoverer
> >>>        baremetal-discoverer
> >>>        discoverer-server
> >>>        ovm-discoverer
> >>>        xenserver-discoverer
> >>>
> >>>
> >>>
> >>> If you look at the baremetal hypervisor plugin that is pretty cross
> >> cutting to
> >>> most of ACS.  So the modules it contributes to are as follows
> >>>
> >>> resources/META-INF/cloudstack/baremetal-storage/spring-context.xml
> >>> resources/META-INF/cloudstack/baremetal-compute/spring-context.xml
> >>> resources/META-INF/cloudstack/baremetal-discoverer/spring-context.xml
> >>> resources/META-INF/cloudstack/core/spring-baremetal-core-context.xml
> >>> resources/META-INF/cloudstack/baremetal-planner/spring-context.xml
> >>> resources/META-INF/cloudstack/baremetal-network/spring-context.xml
> >>>
> >>> So it creates child contextes of storage, compute, network, planner,
> and
> >>> discoverer to add its extensions where it needs to be.  Additionally,
> >> you'll notice,
> >>> it adds some beans to the core context from the file resources/META-
> >>> INF/cloudstack/core/spring-baremetal-core-context.xml.  This is because
> >> it has
> >>> some manager class that is used by multiple contexts.
> >>>
> >>> Frank, I understand the scare that we are going too complex, but do you
> >> have
> >>> some other suggestion?  I don't like the idea of one gigantic spring
> >> context.  So I
> >>> feel I am making it as simple as I can while maintaining some order.
> >> People
> >>> just need to create one or more spring xml files and a properties files
> >> that says
> >>> where to put it in the hierarchy.
> >>>
> >>> Additionally, by putting forcing people to put beans in certains
> modules
> >> it
> >>> forces them to think about what is the role of the code.  For example,
> >> today in
> >>> ACS the *ManagerImpl classes are a huge mess.  They implement too many
> >>> interfaces and the code crosses to many architectural boundaries.  Its
> >> about
> >>> time we start splitting things up to be more maintainable.
> >>>
> >>> If you have some time, please check out what I have on github.
> >>>
> >>> Darren
> >>>
> >>>
> >>> On Wed, Sep 18, 2013 at 1:56 PM, Frank Zhang <Frank.Zhang@citrix.com>
> >>> wrote:
> >>>
> >>>> I am not against boundary, I am just against making things unnecessary
> >>>> complex to enable boundary.
> >>>> If we are going this way, I hope we can make it as much as transparent
> >>>> to developers. That means, as a developer, all a plugin I need to do
> >>>> is 1) provide my separate spring xml 2) inject beans I want (valid
> >>>> beans) in my bean and code business logic 3) compile to a jar and put
> >>>> to some place designated by CloudStack. That's it.
> >>>>
> >>>> I raise this topic because I have seen some projects to create
> >>>> boundary making things horrible complex. And sometimes developers are
> >>>> hurt  by wrong boundaries, as a result, to overcome these limitations
> >>>> people write lots of ugly code which makes thing even worse.
> >>>>
> >>>> However, I am still worry about if we can make things so simpler.
> >>>> For example, we may have an orchestration context that contains major
> >>>> beans needed by almost every plugin,  this context can be easily set
> >>>> as parent context for all plugin contexts when bootstrap. However, if
> >>>> a plugin A needs to access some bean defined in plugin B, given they
> >>>> are sibling, how plugin framework resolves the dependency ?
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
> >>>>> Sent: Wednesday, September 18, 2013 11:53 AM
> >>>>> To: dev@cloudstack.apache.org
> >>>>> Subject: Re: [PROPOSAL] Modularize Spring
> >>>>>
> >>>>> I'm not for OSGi either, but contexts are quite useful and will
lead
> >>>>> to
> >>>> better
> >>>>> things.  First off, we don't want one gigantic spring XML config
> >>>>> file
> >>>> like we have
> >>>>> today.  I think we can all agree on that.  So each plugin will have
> >>>>> to
> >>>> supply its
> >>>>> own XML.  So the obstacles you mention, will largely be just that
> >>>>> for
> >>>> people.
> >>>>>
> >>>>> With Spring it is really simple to just inject dependencies and
> >>>>> cross architectural boundaries.  Its currently everywhere in ACS.
> >>>>> You can't
> >>>> just say
> >>>>> we should review code and make sure nobody doesn't do bad things.
 A
> >>>> little bit
> >>>>> of framework to enforce separation is a good thing.  But I'm
> >>>>> guessing
> >>>> you will
> >>>>> disagree with me there.
> >>>>>
> >>>>> Here are some random points on why contexts are good.  Say I want
to
> >>>>> use Spring AOP or Spring TX in my plugin.  With your own context
you
> >>>>> can
> >>>> ensure
> >>>>> that you won't screw with anybody else code by accidentally having
> >>>>> you pointcut match their bean.  You don't have to worry about bean
> >>>>> name
> >>>> conflicts.
> >>>>> If two config files specify bean X, Spring will gladly just use
the
> >>>>> last
> >>>> one.  I've
> >>>>> already found multiply defined beans in ACS, but things still just
> >>>> happen to work.
> >>>>> Having multiple contexts also defines initialization order.  We
can
> >>>> ensure that
> >>>>> the framework is loaded and ready before child context are loaded
> >>>>> and
> >>>> started.
> >>>>> (we kind of do this today with ComponentLifeCycle, but its a hack
in
> >>>>> my
> >>>> mind).
> >>>>> Additionally, when things start you will know, loading context
> >>>>> "crapping
> >>>> plug X".
> >>>>> If spring fails to initialize, the issue it there.  Today, if spring
> >>>> fails to start, it
> >>>>> could be one of over 500 beans causing the weird problem.  The list
> >>>>> goes
> >>>> on
> >>>>> and on.
> >>>>>
> >>>>> Finally, this is the big one and why I really want contexts.  I
have
> >>>> some notes on
> >>>>> the wiki [1] that you might want to read through.  Basically I want
> >>>>> to
> >>>> get to a
> >>>>> more flexible deployment model that allows both a single monolithic
> >>>>> JVM
> >>>> as
> >>>>> today and also a fully distributed system.  Having contexts in
> >>>>> hierarchy
> >>>> will
> >>>>> enable me to accomplish this.  By selecting which contexts are
> >>>>> loaded at runtime will determine what role the JVM takes on.  The
> >>>>> contexts help
> >>>> people
> >>>>> better understand how the distributed architecture will work too,
> >>>>> when
> >>>> we get
> >>>>> there.
> >>>>>
> >>>>> Frank, trust me, I hate complex things.  I don't want OSGi,
> >>>>> classloader
> >>>> magic,
> >>>>> etc.  But I do like organization and a little bit of framework so
> >>>>> that
> >>>> people don't
> >>>>> accidentally shoot themselves in the foot.  I personally like
> >>>>> knowing
> >>>> that I
> >>>>> couldn't have screwed something up, because the framework won't
even
> >>>> allow
> >>>>> it.  If we separate everything as I want today, and then tomorrow
we
> >>>>> say
> >>>> this is
> >>>>> way too much overhead, moving to a flat context is simple.  Don't
> >>>>> think
> >>>> we are
> >>>>> on some slippery slope to classloaders and dependency hell.
> >>>>>
> >>>>> Darren
> >>>>>
> >>>>> [1]
> >>>>>
> >>>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Nothing+to+see+
> >>>> her
> >>>>> e...#Nothingtoseehere...-DeploymentModels
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Wed, Sep 18, 2013 at 11:22 AM, Frank Zhang
> >>>>> <Frank.Zhang@citrix.com>wrote:
> >>>>>
> >>>>>> What's the point in using separate spring context per plugin?
> >>>>>> Separate class loader is the thing I hate most in OSGI, I am
> >>>>>> afraid we are on the same way.
> >>>>>> Frankly speaking, I never see benefits of this *separate* model,
> >>>>>> our project(or most projects) is not like Chrome which has to
> >>>>>> create sandbox for extensions in order to avoid bad plugin screws
> >>>>>> up the whole browser(however, I still see bad plugins screw
up my
> >>>>>> Chrome well).
> >>>>>> Projects like CloudStack using plugin to decouple architecture
> >>>>>> should not introduce many isolations to plugin writer, the point
> >>>>>> preventing wrong use of some components is not making much sense
> >>>>>> to me. If a plugin not following guide(if we have
> >>>>>> it) we should kick it out, instead of making obstacles for 99%
> >>>>>> good
> >>>> people.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> -----Original Message-----
> >>>>>>> From: Darren Shepherd [mailto:darren.s.shepherd@gmail.com]
> >>>>>>> Sent: Wednesday, September 18, 2013 10:33 AM
> >>>>>>> To: dev@cloudstack.apache.org
> >>>>>>> Subject: Re: [PROPOSAL] Modularize Spring
> >>>>>>>
> >>>>>>> Right, component isn't a thing.  I probably shouldn't use
that
> >> term.
> >>>>>>> I
> >>>>>> want to
> >>>>>>> standarize on the naming convention of plugin, module, and
> >> extension.
> >>>>>> It is
> >>>>>>> explained a bit on the wiki [1] but I'll try to do a little
> >>>>>>> better job
> >>>>>> here.  So a
> >>>>>>> plugin is basically a jar.  A jar contains multiple modules.
 A
> >>>>>>> modules
> >>>>>> ends up
> >>>>>>> being a spring application context that composes multiple
> >>>>>>> configuration
> >>>>>> files.
> >>>>>>> Modules are assembled into a hierarchy at runtime.  Extensions
> >>>>>>> are implementations of interfaces that exist in a module.
 A
> >>>>>>> maven project produces a jar, so a plugin ends up being
a maven
> >>>>>>> project
> >>>> also.
> >>>>>>>
> >>>>>>> So currently today we don't have a strong definition of
Plugin
> >>>>>>> and I
> >>>>>> hope to
> >>>>>>> address that.
> >>>>>>>
> >>>>>>> Darren
> >>>>>>>
> >>>>>>> [1]
> >>>>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-
> >>>>>>> ins%2C+Modules%2C+and+Extensions
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Sep 18, 2013 at 4:25 AM, Daan Hoogland
> >>>>>>> <daan.hoogland@gmail.com>wrote:
> >>>>>>>
> >>>>>>>> sounds great Darren,
> >>>>>>>>
> >>>>>>>> By component, you mean maven project or some larger
chunk like
> >>>>>>>> distribution package? (did i miss this definition somewhere
or
> >>>>>>>> do we define the components now?)
> >>>>>>>>
> >>>>>>>> regards,
> >>>>>>>> Daan
> >>>>>>>>
> >>>>>>>> On Wed, Sep 18, 2013 at 12:10 AM, Darren Shepherd
> >>>>>>>> <darren.s.shepherd@gmail.com> wrote:
> >>>>>>>>> Currently ACS code is fairly modular in that you
can add
> >>>>>>>>> plug-ins to ACS
> >>>>>>>> to
> >>>>>>>>> extend most functionality.  Unfortunately ACS is
not
> >>>>>>>>> packaged in a
> >>>>>>>> modular
> >>>>>>>>> way.  It is still delivered essentially as one large
unit.
> >>>>>>>>> There are
> >>>>>>>> many
> >>>>>>>>> reason for this but one large barrier to modularizing
ACS is
> >>>>>>>>> that the Spring configuration is managed as one
large unit.
> >>>>>>>>>
> >>>>>>>>> I propose that we break apart the Spring XML configuration
> >>>>>>>>> such that each component contributes its own configuration.
> >>>>>>>>> Additionally each component will be loaded into
its own
> >>>>>>>>> Spring ApplicationContext such that its beans will
not
> >>>>>>>>> conflict with the wiring of other beans in ACS.
 This change
> >>>>>>>> will
> >>>>>>>>> lay the foundation for a richer plugin model and
> >>>>>>>>> additionally a more distributed architecture.
> >>>>>>>>>
> >>>>>>>>> The technical details for this proposal can be found
on the
> >>>>>>>>> wiki
> >>>> [1].
> >>>>>>>>>
> >>>>>>>>> Darren
> >>>>>>>>>
> >>>>>>>>> [1]
> >>>>>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modular
> >>>>>>>> ize+
> >>>>>>>> Spri
> >>>>>>>> ng
> >>>>>>>>
> >>>>>>
> >>>>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message