karaf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andreas Pieber <anpie...@gmail.com>
Subject Re: Thoughts about Karaf profiles / releases / growing number of dependencies (Re: AW: AW: Problem when starting camel-cxf in karaf)
Date Fri, 04 Feb 2011 03:34:33 GMT
OK, there are many points in my head about this for some time now and I hope I
get them out straight here. If I'm not able to make any of my points clear don't
hesitate to ask :)

First of all I'm completely with Guillaume. I don't like the way Karaf blows
up. We have a standard and an enterprise distribution and a minimal distribution
and in future one for android... STOP. I think this is not the way Karaf should go.

In addition, tbh I also don't like the way client projects (such as smx) have to
tweak Karaf around to make it runnable. This makes everything complex and not
easy to use understand/start with.

Next I think the root of all evil are our configuration files. The need that we
have to tweak system.properties, custom.properties, jre.properties,
org...features.cfg to get a running, custom distribution. This does not feel
right.

Fourth: I don't like that we need to use our own maven plugins to build Karaf is
IMHO a sign that Karaf is becoming much too big...

Last but not least, I hate that I have to google for all those feature urls 
and collect them locally to not forgetting them :)

OK, so far to the problems. According to the mails I've read by now I think I'm
not the only one aware of those problems :) --> I would propose some roadmap
to get this straight again:

1) branch off karaf-2.x branch now (independent of the missing versions) and
stay as it is now. Some ppl already depend on karaf-2.1.99-SNAPSHOT and I don't
think that we'll make friends if we make any drastically changes to this
version.

2) Start developing karaf-2.99.99-SNAPSHOT right now.

3) As a first step I think we should add something like an online registry to
Karaf where feature files could be found. E.g. something like
karaf.apache.org/registry.xml. This would allow us to make Karaf behave the same
without having to pack everything together during assembly phase. Here it may
also help to add a features:addregistryurl command allowing clients to host
their own registries.

4) Move enterprise.features.xml into aries project, split standard.features.xml
into the different projects: karaf-obr, karaf-ssh, ops4j-web (e.g. with jetty),
spring2, spring3, ... and create own projects for the additional components. I'm
with Jamie that this will complex the release process, but IMHO everything we
create features files for shouldn't be in the core. As said somehow I even don't
like that we have features.xml in the core at all. All those features.xml have
to be added to the registry. This way we could guarantee that Karaf still
behaves the same although we've reduced the core packages/repo/distribution.

5) We have to extend the kar/features.xml model drastically adding variation
points for the different configuration possibilities in Karaf. E.g. we need a
variation point for lib, jre.properties, etc/(to add additional property files),
system (where we drop additional .jre bundles, ...), brandings, ... With that some variation
points require Karaf to reboot (we couldn't change everything at runtime). Which
means we have to cache the changes and apply them on reboot. In addition I know
that there will be some pain in config file merging, but I think this will be
worth the effort.

6) Now that we've removed all the specific things from Karaf we can reduce the
build process completely to one minimal distribution of Karaf (at maximum one
additional for android?!) without using the features concepts directly in the
kernel. This will also allow us to develop the karaf-maven-plugin directly in
the kernel without the burden of forking the process anywhere.

7) I think by that we're almost finished now. We can start contributing
features/kar packages for cxf, apache ds, camel, ... and add them again to the
registry. For this process the karaf-maven-plugin could/should be used.

8) Now a user will first of all create its own kar/features which tweak the
kernel as required. Finally he can either provide the kar/features bundle only.
The assembly process will (and should) always look like (reduced):

karaf-maven-plugin
execution: distribution
kar-list (branding, docs, ... everything is a kar file, nothing more here)

IMHO with all of this we can finally provide only one very small distribution.
Someone only experimenting/developing can do a 

features:install war, cfx, ds
admin:reboot

Everyone else can create distributions as he likes.

OK, I know that not all of those points are new or exclusively by me, some go a
little bit a different direction than discussed by now and not all of them are
nailed down to the latest detail (by now). Please don't flame me because of this;
I know that this is not the only way to go but it somehow feels right to me :)

so... wdyt?

kind regards,
andreas

On Fri, Feb 04, 2011 at 12:07:42AM +0100, Guillaume Nodet wrote:
> Yeah, the problem is that Karaf itself isn't a container for Camel or
> CXF and we have some users that just want to plain JRE without any
> tweaks for running SAAJ because they don't care.
> ServiceMix on the other hand is dedicated to host such applications,
> so that's why the default config works better.
> I'm ok with the idea of profiles in karaf, but we need to make sure we
> don't end up with having to include and depend on all apache projects,
> as Karaf itself has imho no purpose of becoming a default distribution
> of CXF, Camel, ActiveMQ, Directory, etc...
> I think each project could easily provide a karaf features so that
> they would easily be deployed in a bare Karaf, but at some point, it
> does not really scale nor make sense to put everything back into
> Karaf.
> 
> Tweaking the system properties is sometimes needed depending on what
> you need to deploy, because OSGi is lacking on certain areas (or
> because the JVM isn't really modular, depending on how you see
> things).  Some people are happy with using the JAXB implementation
> from the JVM without being able to change it easily, some people may
> want to be able to deploy those as OSGi bundles.  Karaf can't do both
> at the same time.
> 
> Another point, is that the amount of third party dependencies is
> becoming increasingly important in Karaf, and that's really becoming a
> problem for simply releasing Karaf in an efficient manner.
> So I'm tempted to say that we should push back on those projects and
> have them provide karaf features, rather than having karaf providing
> features for those.  I'm mostly thinking here about pax-web and the
> war support (which is really cool, no doubt on that) and aries and
> support for things we don't embed by default (jpa, etc..).
> 
> I certainly don't have a good answer yet, but I just want to make sure
> everyone is aware of the potential problem.
> 
> If we keep adding dependencies, our release cycles will necessarily be
> longer and longer ....  For example camel has a release cycle of one
> minor version every few months, ServiceMix even less, while CXF has
> much less external dependencies and manage to keep a high frequency of
> releases.  2.2 could have been shipped early december, but we were
> waiting for aries.  Now aries has been released, but we still have
> some snapshots dependencies on some felix or ops4j components.  And
> we've added stuff that's not completely stabilized.
> 
> Once again, I'm just trying to state facts so that everybody
> understand the situation.  I'm personnaly not really happy that the
> 2.2 release is planned since 2 months but still can't get out and I
> think it clearly means that we're going in a wrong direction somehow
> ....
> 
> Sorry for the rant.  There are a bunch of unrelated things here, ...
> 
> On Wed, Feb 2, 2011 at 11:29, Jean-Baptiste Onofré <jb@nanthrax.net> wrote:
> > Claus already raised a Jira about that.
> >
> > Guillaume wasn't very ok to set the jre.properties by default.
> >
> > On the other hand, I launched a discussion thread about Karaf profiles. It's
> > typically the kind of tuning which is part of a profiles.
> >
> > Waiting for the profiles design, we can provide a Karaf Enterprise
> > distribution including this kind of tuning related to Camel/CXF/SMX.
> >
> > I add Karaf dev mailing list to see what the team says :)
> >
> > Regards
> > JB
> >
> > On 02/02/2011 11:21 AM, Christian Schneider wrote:
> >>
> >> Are these adapted jre.properties only usefull for Servicemix and the
> >> Talend Service Factory or do you think it would make sense to change them in
> >> the karaf distribution.
> >>
> >> Best regards
> >>
> >> Christian
> >>
> >>
> >> -----Ursprüngliche Nachricht-----
> >> Von: Jean-Baptiste Onofré [mailto:jb@nanthrax.net]
> >> Gesendet: Mittwoch, 2. Februar 2011 11:02
> >> An: users@camel.apache.org
> >> Betreff: Re: AW: Problem when starting camel-cxf in karaf
> >>
> >> Yeah, you can take a look on the ServiceMix one too :)
> >>
> >> Regards
> >> JB
> >>
> >> On 02/02/2011 10:26 AM, Christian Schneider wrote:
> >>>
> >>> I think I was able to solve the problem. I had to change the
> >>> jre.properties to exclude some packages from the system bundle export.
> >>> The jre.properties from the Talend Service Factory seem to work:
> >>>
> >>> http://de.talend.com/products-application-integration/talend-service-factory-community-edition.php
> >>>
> >>> Christian
> >>>
> >>>
> >>> -----Ursprüngliche Nachricht-----
> >>> Von: Christian Schneider [mailto:cschneider@talend.com]
> >>> Gesendet: Mittwoch, 2. Februar 2011 09:52
> >>> An: users@camel.apache.org
> >>> Betreff: Problem when starting camel-cxf in karaf
> >>>
> >>> Hi all,
> >>>
> >>> I am trying to run Apache Camel (feature camel-cxf) in Apache Karaf.
> >>>
> >>> I am using a fresh Karaf 2.1.3 download.
> >>> I start karaf and enter:
> >>>
> >>>> features:addurl
> >>>> mvn:org.apache.camel.karaf/apache-camel/2.6.0/xml/features
> >>>> features:install camel-cxf
> >>>
> >>> I get an exception while starting
> >>> org.apache.servicemix.bundles.saaj-impl:
> >>> Unable to resolve 97.0: missing requirement [97.0] package;
> >>> (package=com.sun.org.apache.xerces.internal.dom)
> >>>
> >>> Any idea what I am doing wrong?
> >>>
> >>> Christian
> >>>
> >>> ----------------------
> >>>
> >>> 02.02.2011 09:46:08 org.apache.karaf.main.SimpleFileLock lock
> >>> INFO: locking
> >>> 09:46:09,765 | INFO  | FelixStartLevel  | jmx
> >>>  | ?                                   ? | 20 - org.apache.aries.jmx
-
> >>> 0.2.0.incubating | Starting JMX OSGi agent
> >>> 09:46:09,781 | INFO  | FelixStartLevel  | jmx
> >>>  | ?                                   ? | 20 - org.apache.aries.jmx
-
> >>> 0.2.0.incubating | Registering MBean with ObjectName
> >>> [osgi.compendium:service=cm,version=1.3] for service with service.id [10]
> >>> 09:46:09,796 | WARN  | rint Extender: 3 | BlueprintContainerImpl
> >>>   | container.BlueprintContainerImpl  252 | 7 - org.apache.aries.blueprint
-
> >>> 0.2.0.incubating | Bundle org.apache.karaf.features.command is waiting for
> >>> namespace handlers
> >>> [(&(objectClass=org.apache.aries.blueprint.NamespaceHandler)(osgi.service.blueprint.namespace=http://karaf.apache.org/xmlns/shell/v1.0.0))]
> >>> 09:46:09,796 | WARN  | rint Extender: 2 | BlueprintContainerImpl
> >>>   | container.BlueprintContainerImpl  252 | 7 - org.apache.aries.blueprint
-
> >>> 0.2.0.incubating | Bundle org.apache.karaf.shell.osgi is waiting for
> >>> namespace handlers
> >>> [(&(objectClass=org.apache.aries.blueprint.NamespaceHandler)(osgi.service.blueprint.namespace=http://karaf.apache.org/xmlns/shell/v1.0.0))]
> >>> 09:46:09,796 | WARN  | JMX OSGi Agent   | jmx
> >>>  | ?                                   ? | 20 - org.apache.aries.jmx
-
> >>> 0.2.0.incubating | There are no MBean servers registred, can't register
> >>> MBeans
> >>> 09:46:09,890 | WARN  | rint Extender: 2 | BlueprintContainerImpl
> >>>   | container.BlueprintContainerImpl  252 | 7 - org.apache.aries.blueprint
-
> >>> 0.2.0.incubating | Bundle org.apache.karaf.shell.console is waiting for
> >>> namespace handlers
> >>> [(&(objectClass=org.apache.aries.blueprint.NamespaceHandler)(osgi.service.blueprint.namespace=http://aries.apache.org/blueprint/xmlns/blueprint-ext/v1.0.0))]
> >>> 09:46:09,968 | WARN  | rint Extender: 2 | BlueprintContainerImpl
> >>>   | container.BlueprintContainerImpl  252 | 7 - org.apache.aries.blueprint
-
> >>> 0.2.0.incubating | Bundle org.apache.karaf.shell.dev is waiting for
> >>> namespace handlers
> >>> [(&(objectClass=org.apache.aries.blueprint.NamespaceHandler)(osgi.service.blueprint.namespace=http://karaf.apache.org/xmlns/shell/v1.0.0))]
> >>> 09:46:10,218 | WARN  | rint Extender: 3 | BlueprintContainerImpl
> >>>   | container.BlueprintContainerImpl  252 | 7 - org.apache.aries.blueprint
-
> >>> 0.2.0.incubating | Bundle org.apache.karaf.shell.ssh is waiting for
> >>> namespace handlers
> >>> [(&(objectClass=org.apache.aries.blueprint.NamespaceHandler)(osgi.service.blueprint.namespace=http://karaf.apache.org/xmlns/shell/v1.0.0))]
> >>> 09:46:10,234 | WARN  | rint Extender: 3 | BlueprintContainerImpl
> >>>   | container.BlueprintContainerImpl  252 | 7 - org.apache.aries.blueprint
-
> >>> 0.2.0.incubating | Bundle org.apache.karaf.admin.command is waiting for
> >>> namespace handlers
> >>> [(&(objectClass=org.apache.aries.blueprint.NamespaceHandler)(osgi.service.blueprint.namespace=http://karaf.apache.org/xmlns/shell/v1.0.0))]
> >>> 09:46:10,250 | WARN  | rint Extender: 3 | BlueprintContainerImpl
> >>>   | container.BlueprintContainerImpl  252 | 7 - org.apache.aries.blueprint
-
> >>> 0.2.0.incubating | Bundle org.apache.karaf.shell.commands is waiting for
> >>> namespace handlers
> >>> [(&(objectClass=org.apache.aries.blueprint.NamespaceHandler)(osgi.service.blueprint.namespace=http://karaf.apache.org/xmlns/shell/v1.0.0))]
> >>> 09:46:10,375 | WARN  | rint Extender: 2 | BlueprintContainerImpl
> >>>   | container.BlueprintContainerImpl  252 | 7 - org.apache.aries.blueprint
-
> >>> 0.2.0.incubating | Bundle org.apache.karaf.shell.packages is waiting for
> >>> namespace handlers
> >>> [(&(objectClass=org.apache.aries.blueprint.NamespaceHandler)(osgi.service.blueprint.namespace=http://karaf.apache.org/xmlns/shell/v1.0.0))]
> >>> 09:46:11,609 | INFO  | Thread-5         | FeaturesServiceImpl
> >>>  | res.internal.FeaturesServiceImpl  293 | 19 -
> >>> org.apache.karaf.features.core - 2.1.3 | Bundles to refresh:
> >>> 09:46:11,640 | INFO  | rint Extender: 3 | SecurityUtils
> >>>  | e.sshd.common.util.SecurityUtils   80 | 25 - sshd-core - 0.4.0 |
> >>> BouncyCastle not registered, using the default JCE provider
> >>> 09:46:11,968 | INFO  | JMX OSGi Agent   | jmx
> >>>  | ?                                   ? | 20 - org.apache.aries.jmx
-
> >>> 0.2.0.incubating | Registering org.osgi.jmx.framework.BundleStateMBean to
> >>> MBeanServer com.sun.jmx.mbeanserver.JmxMBeanServer@6e3d60 with name
> >>> osgi.core:type=bundleState,version=1.5
> >>> 09:46:12,000 | INFO  | JMX OSGi Agent   | jmx
> >>>  | ?                                   ? | 20 - org.apache.aries.jmx
-
> >>> 0.2.0.incubating | Registering org.osgi.jmx.framework.ServiceStateMBean
to
> >>> MBeanServer com.sun.jmx.mbeanserver.JmxMBeanServer@6e3d60 with name
> >>> osgi.core:type=serviceState,version=1.5
> >>> 09:46:12,000 | INFO  | JMX OSGi Agent   | jmx
> >>>  | ?                                   ? | 20 - org.apache.aries.jmx
-
> >>> 0.2.0.incubating | Registering org.osgi.jmx.framework.FrameworkMBean to
> >>> MBeanServer com.sun.jmx.mbeanserver.JmxMBeanServer@6e3d60 with name
> >>> osgi.core:type=framework,version=1.5
> >>> 09:46:12,031 | INFO  | JMX OSGi Agent   | jmx
> >>>  | ?                                   ? | 20 - org.apache.aries.jmx
-
> >>> 0.2.0.incubating | Registering
> >>> org.osgi.jmx.service.cm.ConfigurationAdminMBean to MBeanServer
> >>> com.sun.jmx.mbeanserver.JmxMBeanServer@6e3d60 with name
> >>> osgi.compendium:service=cm,version=1.3
> >>> 09:46:12,031 | INFO  | JMX OSGi Agent   | jmx
> >>>  | ?                                   ? | 20 - org.apache.aries.jmx
-
> >>> 0.2.0.incubating | Registering org.osgi.jmx.framework.PackageStateMBean
to
> >>> MBeanServer com.sun.jmx.mbeanserver.JmxMBeanServer@6e3d60 with name
> >>> osgi.core:type=packageState,version=1.5
> >>> 09:48:08,234 | INFO  | l Console Thread | FeaturesServiceImpl
> >>>  | res.internal.FeaturesServiceImpl  293 | 19 -
> >>> org.apache.karaf.features.core - 2.1.3 | Bundles to refresh:
> >>> 09:48:08,656 | INFO  | l Console Thread | ContextLoaderListener
> >>>  | .activator.ContextLoaderListener  356 | 44 -
> >>> org.springframework.osgi.extender - 1.2.0 | Starting
> >>> [org.springframework.osgi.extender] bundle v.[1.2.0]
> >>> 09:48:08,781 | INFO  | l Console Thread | ExtenderConfiguration
> >>>  | al.support.ExtenderConfiguration  150 | 44 -
> >>> org.springframework.osgi.extender - 1.2.0 | No custom extender configuration
> >>> detected; using defaults...
> >>> 09:48:08,781 | INFO  | l Console Thread | TimerTaskExecutor
> >>>  | heduling.timer.TimerTaskExecutor  106 | 38 - org.springframework.context
> >>> - 3.0.5.RELEASE | Initializing Timer
> >>> 09:48:09,281 | INFO  | l Console Thread | Activator
> >>>  | apache.camel.impl.osgi.Activator   83 | 51 - org.apache.camel.camel-core
> >>> - 2.6.0 | Camel activator starting
> >>> 09:48:09,312 | INFO  | l Console Thread | Activator
> >>>  | apache.camel.impl.osgi.Activator   86 | 51 - org.apache.camel.camel-core
> >>> - 2.6.0 | Camel activator started
> >>> 09:48:10,968 | INFO  | l Console Thread | Activator
> >>>  | apache.camel.impl.osgi.Activator   90 | 51 - org.apache.camel.camel-core
> >>> - 2.6.0 | Camel activator stopping
> >>> 09:48:10,968 | INFO  | l Console Thread | Activator
> >>>  | apache.camel.impl.osgi.Activator   92 | 51 - org.apache.camel.camel-core
> >>> - 2.6.0 | Camel activator stopped
> >>> 09:48:11,984 | INFO  | l Console Thread | ContextLoaderListener
> >>>  | .activator.ContextLoaderListener  449 | 44 -
> >>> org.springframework.osgi.extender - 1.2.0 | Stopping
> >>> [org.springframework.osgi.extender] bundle v.[1.2.0]
> >>> 09:48:11,984 | INFO  | l Console Thread | TimerTaskExecutor
> >>>  | heduling.timer.TimerTaskExecutor  179 | 38 - org.springframework.context
> >>> - 3.0.5.RELEASE | Cancelling Timer
> >>> 09:48:12,281 | INFO  | l Console Thread | Console
> >>>  | araf.shell.console.jline.Console  188 | 11 -
> >>> org.apache.karaf.shell.console - 2.1.3 | Exception caught while executing
> >>> command
> >>> java.lang.Exception: Could not start bundle
> >>> mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.saaj-impl/1.3.2_1
> >>> in feature(s) camel-cxf-2.6.0: Unresolved constraint in bundle
> >>> org.apache.servicemix.bundles.saaj-impl [97]: Unable to resolve 97.0:
> >>> missing requirement [97.0] package;
> >>> (package=com.sun.org.apache.xerces.internal.dom)
> >>>                  at
> >>> org.apache.karaf.features.internal.FeaturesServiceImpl.installFeatures(FeaturesServiceImpl.java:326)[19:org.apache.karaf.features.core:2.1.3]
> >>>                  at
> >>> org.apache.karaf.features.internal.FeaturesServiceImpl.installFeature(FeaturesServiceImpl.java:254)[19:org.apache.karaf.features.core:2.1.3]
> >>>                  at
> >>> org.apache.karaf.features.internal.FeaturesServiceImpl.installFeature(FeaturesServiceImpl.java:250)[19:org.apache.karaf.features.core:2.1.3]
> >>>                  at
> >>> org.apache.karaf.features.command.InstallFeatureCommand.doExecute(InstallFeatureCommand.java:51)[9:org.apache.karaf.features.command:2.1.3]
> >>>                  at
> >>> org.apache.karaf.features.command.FeaturesCommandSupport.doExecute(FeaturesCommandSupport.java:39)[9:org.apache.karaf.features.command:2.1.3]
> >>>                  at
> >>> org.apache.karaf.shell.console.OsgiCommandSupport.execute(OsgiCommandSupport.java:38)[11:org.apache.karaf.shell.console:2.1.3]
> >>>                  at
> >>> org.apache.felix.gogo.commands.basic.AbstractCommand.execute(AbstractCommand.java:35)[11:org.apache.karaf.shell.console:2.1.3]
> >>>                  at
> >>> org.apache.felix.gogo.runtime.shell.CommandProxy.execute(CommandProxy.java:50)[11:org.apache.karaf.shell.console:2.1.3]
> >>>                  at
> >>> org.apache.felix.gogo.runtime.shell.Closure.execute(Closure.java:229)[11:org.apache.karaf.shell.console:2.1.3]
> >>>                  at
> >>> org.apache.felix.gogo.runtime.shell.Closure.executeStatement(Closure.java:162)[11:org.apache.karaf.shell.console:2.1.3]
> >>>                  at
> >>> org.apache.felix.gogo.runtime.shell.Pipe.run(Pipe.java:101)[11:org.apache.karaf.shell.console:2.1.3]
> >>>                  at
> >>> org.apache.felix.gogo.runtime.shell.Closure.execute(Closure.java:79)[11:org.apache.karaf.shell.console:2.1.3]
> >>>                  at
> >>> org.apache.felix.gogo.runtime.shell.CommandSessionImpl.execute(CommandSessionImpl.java:71)[11:org.apache.karaf.shell.console:2.1.3]
> >>>                  at
> >>> org.apache.karaf.shell.console.jline.Console.run(Console.java:170)[11:org.apache.karaf.shell.console:2.1.3]
> >>>                  at java.lang.Thread.run(Thread.java:662)[:1.6.0_23]
> >>> Caused by: org.osgi.framework.BundleException: Unresolved constraint in
> >>> bundle org.apache.servicemix.bundles.saaj-impl [97]: Unable to resolve 97.0:
> >>> missing requirement [97.0] package;
> >>> (package=com.sun.org.apache.xerces.internal.dom)
> >>>                  at
> >>> org.apache.felix.framework.Felix.resolveBundle(Felix.java:3409)[org.apache.felix.framework-3.0.2.jar:]
> >>>                  at
> >>> org.apache.felix.framework.Felix.startBundle(Felix.java:1709)[org.apache.felix.framework-3.0.2.jar:]
> >>>                  at
> >>> org.apache.felix.framework.BundleImpl.start(BundleImpl.java:905)[org.apache.felix.framework-3.0.2.jar:]
> >>>                  at
> >>> org.apache.felix.framework.BundleImpl.start(BundleImpl.java:892)[org.apache.felix.framework-3.0.2.jar:]
> >>>                  at
> >>> org.apache.karaf.features.internal.FeaturesServiceImpl.installFeatures(FeaturesServiceImpl.java:323)[19:org.apache.karaf.features.core:2.1.3]
> >>>                  ... 14 more
> >>>
> >
> 
> 
> 
> -- 
> Cheers,
> Guillaume Nodet
> ------------------------
> Blog: http://gnodet.blogspot.com/
> ------------------------
> Open Source SOA
> http://fusesource.com

Mime
View raw message