karaf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Achim Nierbeck <bcanh...@googlemail.com>
Subject Re: [HEADS UP] Profiles and static distributions
Date Mon, 15 Dec 2014 09:42:53 GMT
Hi JB,

sounds like a good solution, this way we add some more value to Cave. And
everything is provided from the same technology, so no breach there
especially concerning the git/gist/external repo thing.

+1

regards, Achim


2014-12-15 10:34 GMT+01:00 Jean-Baptiste Onofré <jb@nanthrax.net>:

> Hi guys,
>
> following Guillaume work on the profile, I have an extended proposal that
> I want to discuss with you.
>
> The Karaf profiles have to be stored somewhere: on github/git for
> instance, in order to be applied on a distribution.
> For instance, what I tested is the karaf docker.io feature with profiles:
> it works fine, the docker.io feature bootstraps a docker image with
> Karaf, and I applied a profile on this Karaf instance.
>
> A storage is also needed for the docker.io feature: where the images are
> stored (basically a docker.io github like).
>
> On a private infrastructure, companies don't want to use "external"
> storage (like github): they want to use first their "local" storages. It's
> also valid for hybrid infrastructure: their want to use their local storage
> first (as master), and possible proxy external storages.
>
> Right now, we have Karaf Cave providing:
> - Maven repositories management (stored in Cave or mirroring/proxying
> external repositories)
> - OBR server management
> - Feature Enterprise Repositories management
> Cave is especially interesting with Cellar where all cluster nodes share
> Cave repositories.
>
> In order to have centralised storage (and possibly replicated storage), I
> wonder if we can not extend Cave to provide:
> - Karaf profiles repositories
> - Docker.io images repositories
> It could work like for Maven repository, meaning local storage or
> mirror/proxy to another one.
>
> Like this, we could have Karaf and shell commands to bootstrap Karaf
> docker.io images, or apply profiles on an instance.
>
> WDYT ?
>
> Regards
> JB
>
> On 12/11/2014 09:04 AM, Guillaume Nodet wrote:
>
>> I've just committed to the 4.x branch two new features:
>>    * profiles
>>    * static distributions
>>
>> Profiles
>> ======
>>
>> A profile is a data structure that brings together configuration and
>> provisioning, so it's related to features, but is quite different.  A demo
>> is available at [1].
>> A profile contains a list of configuration files which can be either
>> properties based or in any other format.  Properties based configurations
>> are handled in a very specific way, as they can overlay each other (more
>> below).
>> One of this properties based configuration holds provisioning informations
>> along with profiles specific attributes, such as a list of parent
>> profiles,
>> a list of bundles, a list of features repositories and a list of features.
>>
>> A profile can have one or more parents, so the hierarchy is an inverted
>> tree.  This allows to define generic profiles and specialise them in
>> children.
>>
>> Overlay profile: at some point, there is a need to flatten this hierarchy
>> into an "overlay" profile.  This process is done by walking into the
>> parents hierarchy (depth first) and computing the resulting configuration
>> for each configuration file.  Non properties configuration files simply
>> overwrite each other, while properties based configuration complement each
>> other (with the ability to clear a single value or all values).
>>
>> Effective profile: properties configurations can contain placeholders to
>> resolve. The only one defined atm is a property placeholder pointing to a
>> value in a different configuration.  Those placeholders are resolved on
>> the
>> overlay profile to give the end result effective profile which will be
>> actually used.  An example of using this placeholder is shown in [2] which
>> end up pointing to the configuration in its parent [3]
>>
>> Provisioning: each profile contains a list of feature repositories,
>> features and bundles.  Those information will be used in the effective
>> profile to know which bundles and features are needed on a given instance.
>>    Profiles are not used at runtime for the moment and there are 2 main
>> usage : creating child instances and generating distributions using the
>> maven plugin.
>>
>> Profile manipulation : profiles are manipulated using the Profiles class
>> (static helpers) and the ProfileBuilder (to build profile instances, which
>> are immutable).  Reading and writing profiles is done using the jdk7 in
>> FileSystem / Path interface, which obviously provides support for the
>> standard file system, but could be used with various file systems (inside
>> a
>> zip [4], readonly github remote repository [5], sftp [6], in memory [7].
>> Those are the ones I know about).  A set of commands can be used to edit
>> the profiles.
>>
>> Further considerations: possible improvements include an overlay mechanism
>> for xml, integration with cellar, file systems using a real git repo,
>> hazelcast, zookeeper, etc...  Applying profiles at runtime could be done
>> too.   Better integration with kars could be provided too.
>>
>>
>> Static distributions
>> ==============
>>
>> One use case of profiles is to generate distributions.  Changes on the
>> profile need a rebuild on the distribution (this may or may not be a
>> problem, depending on your environment, but if it can seems weird in an
>> OSGi world, it's not so much in a cloud environment). This leads to the
>> distribution being mostly static, i.e. the user should not manually
>> install
>> / uninstall stuff.
>> A demo is available at [8]
>> Based on this idea, I improved the maven plugin a bit to allow generating
>> mostly static distribution : the profiles / features are all installed in
>> etc/startup.properties.  A simplistic read-only configuration admin is
>> used
>> that will pick up configuration in the etc/ folder directly.  This lead to
>> no runtime dependencies for the provisioning of karaf itself :
>> fileinstall,
>> the features service can all be removed.  The bare framework can only
>> contain 3 bundles : pax-logging + the static configadmin [9].  One
>> improvement is that the distribution is generated using reference:file:
>> urls in startup.properties, avoiding a copy of each jar into the cache.
>> An
>> additional improvement could be to pre-compute the wiring which can take
>> some time (but this is not supported by felix).
>>
>>
>> Thoughts welcomed !
>>
>>
>> [1] https://github.com/apache/karaf/tree/master/demos/profiles
>> [2]
>> https://github.com/apache/karaf/blob/master/demos/
>> profiles/registry/src/main/resources/karaf.profile/profile.cfg#L25
>> [3]
>> https://github.com/apache/karaf/blob/master/demos/
>> profiles/registry/src/main/resources/default.profile/version.cfg
>> [4]
>> http://docs.oracle.com/javase/7/docs/technotes/guides/io/
>> fsp/zipfilesystemprovider.html
>> [5] https://github.com/gnodet/githubfs
>> [6]
>> https://github.com/gnodet/mina-sshd/commit/f68468c686099f6f12a9093e949c09
>> e16f618704
>> [7] https://github.com/google/jimfs
>> [8] https://github.com/apache/karaf/blob/master/demos/profiles/static/
>> [9]
>> https://github.com/apache/karaf/blob/master/demos/
>> profiles/static/src/main/resources/features.xml
>>
>>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>



-- 

Apache Member
Apache Karaf <http://karaf.apache.org/> Committer & PMC
OPS4J Pax Web <http://wiki.ops4j.org/display/paxweb/Pax+Web/> Committer &
Project Lead
blog <http://notizblog.nierbeck.de/>
Co-Author of Apache Karaf Cookbook <http://bit.ly/1ps9rkS>

Software Architect / Project Manager / Scrum Master

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message