ace-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthew Bishop <m...@thebishops.org>
Subject Re: Where does Ace server store everything?
Date Wed, 07 Aug 2013 13:30:23 GMT
We have two problems here: definition of artifact / feature / distributions
(AFD?), and ACE Server loadout of AFDs.

*AFD definitions:* it makes a lot of sense to me to use the Maven project
file structure to declare which bundles that define the AFD components.
Maven is a well-known and capable at this task and has considerable tooling
for transitive dependency declaration and discovery. I can imagine the
following design:

*Artifact* - maven jar artifact
*Feature* - POM that lists <dependencies> on artifacts. Builds out a pom
artifact.
*Distribution* - POM that lists <dependencies> on features. Builds out a
pom artifact.

I know you might be thinking that maven isn't good enough for OSGi because
it doesn't understand the OSGi manifest headers, or OBR, or capabilities,
and you are right. However, at the end of the day, what server boffins want
to do is define every single bundle artifact in a way that can be
understood and managed in a familiar technology. Maven fits this bill very
well.

Most of the tech in OSGi that you know and love at dev time, like BND
Tools, is still applicable. All I am suggesting is that the AFD definitions
use the maven POM files, not necessarily the 'mvn' runtimes to generate the
actual bundle jars. Using maven like this will populate the maven
repositories with AFD definitions that will be useful for loadout.


*Server Loadout:* The ACE Server (not the targets) would be able to consume
the various Feature and Distribution POMs. It would be configured to look
to maven repositories (like maven central) to load in the bundles as
artifacts. The versioning of the AFDs would be managed by the maven version
numbering itself. Again, very familiar, trusted tech. Easy to roll back if
all hell breaks loose. Easy to write scripts to automate.

The mountain I am really trying to climb here is to provide a distribution
package for OSGi applications, a package that can be understood and
examined in existing and well-integrated deployment technology.


On Tue, Aug 6, 2013 at 11:25 PM, Bram de Kruijff <bdekruijff@gmail.com>wrote:

> Hi Matthew,
>
> On Wed, Aug 7, 2013 at 1:06 AM, Matthew Bishop <matt@thebishops.org>
> wrote:
> > Hmm, this looks like a lot like nimble's scripting.
> >
> > Can ACE generate a deployment script that loads up a set of
> > artifacts/features/etc? It would go a long ways towards the
> checkin-ability
> > of what is manipulated in the UI / API.
> >
>
> Generating scripts does not feel like the way to go for me. In this
> example script the artifacts come from a declarative model (the R5
> repository representation), but obviously the association, feature and
> distribution do not. As you already state this information should be
> easy enough to capture in a declarative manner. You would need to do
> this anyway if you would want to generate a script so why not support
> that from ACE directly? I am curious to hear what your idea of the
> 'input format' to a generic import would ideally be.
>
> As a reference, some time ago in wrote an Amdatu ACE Client [0]
> (source repo not online) that talks to the REST API and is also
> capable of doing basic import/export based on a simple JSON
> representation of the custom java model that is in there. However,
> that client does not talk to the OBR, so no uploads. In fact the ACE
> repository uses a similar XML representation for disk storage just
> listing the different entities. Also something we could leverage.
>
> Another idea that I am toying with is to leverage the OSGi Resource
> model of the R5 Repository which would allow us to store all this
> information in a R5 repository. Features and distributions could be
> modeled as specific resource types, providing meta-data
> (attributes/tags) through capabilities and modelling association
> through requirements. This approach would allow capturing all
> information in one single source and standard representation.
>
> So I think there are options :) Again I would like to hear what your
> preferred approach would be an maybe we can collaborate on coming up
> with a generic solution.
>
> Best Regards,
> Bram
>
> [0] http://repository.amdatu.org/release-util/org.amdatu.ace.client/
>
> >
> > On Tue, Aug 6, 2013 at 2:34 PM, Marcel Offermans <
> > marcel.offermans@luminis.nl> wrote:
> >
> >> Hello Matthew,
> >>
> >> On Aug 6, 2013, at 23:10 PM, Matthew Bishop <matt@thebishops.org>
> wrote:
> >>
> >> > On Tue, Aug 6, 2013 at 1:58 PM, Marcel Offermans <
> >> > marcel.offermans@luminis.nl> wrote:
> >> >
> >> >> To give you another example, we recently automated our continuous
> build
> >> >> system so for each succesful build it takes the artifacts that come
> out
> >> of
> >> >> that build and (through a GoGo Shell script) upload them to a
> repository
> >> >> and then create all the metadata in ACE. The end result is that we
> can
> >> >> automatically deploy such code directly to our QA and demo
> environments,
> >> >> completely automated.
> >> >
> >> > Is this gogo script checked into ACE somewhere? It is definitely an
> >> > interesting part of the story. I am working my backwards from the end
> >> > deployment story to the developer, as we did it developer-first and
> ended
> >> > up with pax and maven on the production server. Not cool.
> >>
> >> It is not, because it is specific to our environment. The GoGo commands
> >> are something we only very recently added, so they're not documented
> well
> >> yet.
> >>
> >> However, I have adapted the script we use a bit (taking out the specific
> >> bits). It probably requires some more context to be fully understood,
> and
> >> probably Bram can answer some more detailed questions about it (he wrote
> >> the script). Basically it takes the artifacts from a build, checks if
> they
> >> are really different from the ones we already deployed the previous
> time,
> >> uploads all changed artifacts and configures ACE. We run this script
> with a
> >> client jar from the ACE distribution, and configure it as the startup
> >> script. As soon as the script is done, we shutdown this client again.
> >>
> >>
> >> #
> >> # Continuous deployment gogo script
> >> #
> >>
> >> # environment variables we import here
> >> targetrepourl = "$DEPLOYMENT_TARGETOBR_URL"
> >> releaserepourl = "$DEPLOYMENT_RELEASEOBR_URL"
> >> autoconfurl = "$DEPLOYMENT_AUTOCONF_URL"
> >>
> >> sourceindex = (repo:index /tmp/ace-cdeploy)
> >> sourcerepo  = (repo:repo R5 $sourceindex)
> >> targetrepo = (repo:repo OBR $targetrepourl)
> >> releaserepo = (repo:repo OBR $releaserepourl)
> >>
> >> echo "Deploying release resources"
> >> deployed = repo:cd $releaserepo $sourcerepo $targetrepo
> >>
> >> echo "Ensuring autoconf resource"
> >> autoconf = (coll:first ($targetrepo find
> >> "(osgi.identity=org.apache.felix.deployment.rp.autoconf)"))
> >> if { $autoconf } { echo " -> exists" } \
> >> {
> >>   repo:d $targetrepo $autoconfurl
> >>   autoconf = (coll:first ($targetrepo find
> >> "(osgi.identity=org.apache.felix.deployment.rp.autoconf)"))
> >>   echo " -> deployed"
> >> }
> >>
> >> echo "Opening client workspace"
> >> workspace = (ace:cw)
> >>
> >> echo "Ensuring cdtarget exists"
> >> target = (coll:first ($workspace lt "(id=cdtarget)"))
> >> if { $target } { echo " -> exists" } \
> >> {
> >>   $workspace ct "cdtarget";
> >>   target = (coll:first ($workspace lt "(id=cdtarget)"))
> >>   echo " -> created"
> >> }
> >>
> >> echo "Ensuring cdtarget is registered"
> >> if { $target isRegistered } { } { $target register }
> >>
> >> echo "Ensuring cdtarget is !autoapprove"
> >> $target setAutoApprove false
> >>
> >> # here we set some specific properties used by our configurations
> >> (anonymized)
> >> $target addTag "domain" "foo.com"
> >> $target addTag "ip-address" "127.0.0.1"
> >> $target addTag "timezone" "Europe/Amsterdam"
> >>
> >> echo "Ensuring cdfeature exists"
> >> if { (coll:first ($workspace lf "(name=cdfeature)")) } { echo " ->
> exists"
> >> } { $workspace cf "cdfeature"; echo " -> created" }
> >>
> >> echo "Ensuring cddistribution exists"
> >> if { (coll:first ($workspace ld "(name=cddistribution)")) } { echo " ->
> >> exists" } { $workspace cd "cddistribution"; echo " -> created" }
> >>
> >> echo "Ensuring f2d association"
> >> if { (coll:first ($workspace lf2d "(leftEndpoint=*name=cdfeature*)")) }
> {
> >> echo " -> exists" } { $workspace cf2d "(name=cdfeature)"
> >> "(name=cddistribution)" "1" "1"; echo " -> created" }
> >>
> >> echo "Ensuring a2f association"
> >> if { (coll:first ($workspace la2f "(rightEndpoint=*name=cdfeature*)"))
> } {
> >> echo " -> exists" } { $workspace ca2f "(cdartifact=true)"
> >> "(name=cdfeature)" "10000" "1"; echo " -> created" }
> >>
> >> echo "Ensuring d2t association"
> >> if { (coll:first ($workspace ld2t
> "(leftEndpoint=*name=cddistribution*)"))
> >> } { echo " -> exists" } \
> >>   { $workspace cd2t "(name=cddistribution)" "(id=*)" "1" "1000"; echo "
> ->
> >> created" }
> >>
> >> echo "Ensuring autoconf artifact"
> >> if { (coll:first ($workspace lrp)) } { echo " -> exists" } \
> >> {
> >>   identity = $autoconf getIdentity
> >>   version = $autoconf getVersion
> >>   name = "$identity - $version"
> >>   url = $autoconf getUrl
> >>   mimetype = $autoconf getMimetype
> >>   $workspace ca [ artifactName="$name" url="$url" mimetype="$mimetype"
> >> Bundle-SymbolicName="$identity" Bundle-Version="$version"
> >> Deployment-ProvidesResourceProcessor="org.osgi.deployment.rp.autoconf" ]
> >>   echo " -> created"
> >> }
> >>
> >> echo "Purging old artifacts"
> >> artifacts = ($workspace la)
> >> each $artifacts {
> >>   $workspace da $it
> >> }
> >>
> >> echo "Creating new artifacts"
> >> each $deployed {
> >>   identity = $it getIdentity
> >>   version = $it getVersion
> >>   name = "$identity - $version"
> >>   url = $it getUrl
> >>   mimetype = $it getMimetype
> >>   if { $mimetype equals "application/xml:osgi-autoconf" } \
> >>   { $workspace ca [ artifactName="$name" url="$url" mimetype="$mimetype"
> >> filename="$name" processorPid="org.osgi.deployment.rp.autoconf" ] [
> >> cdartifact="true" ] } \
> >>   { $workspace ca [ artifactName="$name" url="$url" mimetype="$mimetype"
> >> Bundle-SymbolicName="$identity" Bundle-Version="$version" ] [
> >> cdartifact="true"] }
> >> }
> >>
> >> echo "Ensuring approval on target"
> >> if { $target needsApprove } { echo " approving"; $target approve } {
> echo
> >> " no changes" }
> >>
> >> echo "Committing workspace"
> >> $workspace commit
> >>
> >> # Let events settle
> >> misc:sleep 2000
> >> ace:rw $workspace
> >>
> >> # Let events settle
> >> misc:shutdown 2000
> >>
> >>
> >> Greetings, Marcel
> >>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message