camel-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject [CONF] Apache Camel > Camel 3.0 - Ideas
Date Tue, 05 Feb 2013 19:37:00 GMT
    <base href="">
            <link rel="stylesheet" href="/confluence/s/2042/9/1/_/styles/combined.css?spaceKey=CAMEL&amp;forWysiwyg=true"
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
    <h2><a href="">Camel
3.0 - Ideas</a></h2>
    <h4>Page <b>edited</b> by             <a href="">Christian
                         <h4>Changes (1)</h4>
<div id="page-diffs">
                    <table class="diff" cellpadding="0" cellspacing="0">
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >It would be good to have a camel-java8-dsl
component that offers a JDK8 DSL which uses all the nice new stuff from JDK8 with higher order
functions, closures, et all. <br>Though this may comes later in Camel 3.x when JDK8
is GA. <br></td></tr>
            <tr><td class="diff-added-lines" style="background-color: #dfd;">At
least stuff like Predicate, Expression, AggregationStrategy etc. are &quot;functional
interfaces&quot; (containing only one method) and Java 8 applications can implement them
using lambdas. That&#39;s only a start, but it doesn&#39;t require a specific DSL.
            <tr><td class="diff-unchanged" > <br> <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
    </div>                            <h4>Full Content</h4>
                    <div class="notificationGreySide">
        <h2><a name="Camel3.0-Ideas-Camel3.0Ideas"></a>Camel 3.0 Ideas</h2>

<div class='panelMacro'><table class='warningMacro'><colgroup><col width='24'><col></colgroup><tr><td
valign='top'><img src="/confluence/images/icons/emoticons/forbidden.gif" width="16"
height="16" align="absmiddle" alt="" border="0"></td><td><b>WIP</b><br

<p>Camel is now almost 6 years old and its second revision camel-2.x is more than 4.5
years old already. Camel is extremely mature, used in production by a large number of organizations
from small to large and even governments. We feel like we really hit the initial target of
simplifying integration. Camel's middleware abstraction api and the eip based routing brought
a lot of positive feedback from users.</p>

<p>There is however more that could be done to simplify the work of integration developers
who need new components (not shipped with camel for licensing - copyleft of commercial - or
other reasons) or new integration patterns or algorithms or even new tools. We learned a lot
in the past years and benefited from a strong and continuously growing community. It's time
to put what we learned to good use and re-engineer your favourite integration framework yet

<p>The middleware abstractions look pretty solid, and aside from some possible reshuffling
we don't expect major changes. As a consequence, most of the components will retain the same
general feel. The core will however be rearchitected to become even more pluggable and modular.
We will however spare no effort to make a new Camel 3 be as backward compatible as possible
and when not possible at least provide a painless migration path.</p>

<p>This is a mindmap of ideas for improving Camel 3.0. Fell free to discuss this on
the Camel <a href="/confluence/display/CAMEL/Mailing+Lists" title="Mailing Lists">Mailing
Lists</a> if you have other ideas or feedback.</p>

<h3><a name="Camel3.0-Ideas-JDKsupport"></a>JDK support</h3>

<p>We should drop support for JDK6, and require JDK7 as minimim version. eg build and
compile the release with JDK7.<br/>
We should aim to be compatible with JDK8.</p>

<h3><a name="Camel3.0-Ideas-JDK8JavaDSL"></a>JDK8 Java DSL</h3>

<p>It would be good to have a camel-java8-dsl component that offers a JDK8 DSL which
uses all the nice new stuff from JDK8 with higher order functions, closures, et all.<br/>
Though this may comes later in Camel 3.x when JDK8 is GA.<br/>
At least stuff like Predicate, Expression, AggregationStrategy etc. are "functional interfaces"
(containing only one method) and Java 8 applications can implement them using lambdas. That's
only a start, but it doesn't require a specific DSL.</p>

<h3><a name="Camel3.0-Ideas-ClearerArchitectureofCamelCore"></a>Clearer
Architecture of Camel Core</h3>

	<li>The camel components should know as little as possible about camel core</li>
	<li>The classes needed to setup camel should be separate from the things needed at
run time</li>
	<li>Camel Core should be tiny as possible and only contain what really is core</li>

<p>So why should this be important? Currently components depend on camel-core as a whole
and there are no further rules which classes the components should use and which classes should
be private to core. Even classes from the impl package are needed. So this means that any
refactoring we do in camel core could affect all components. As camel is growing steadily
this can become quite problematic.</p>

<h4><a name="Camel3.0-Ideas-Splitcamelcoreintomultipleparts%28hadrian%29"></a>Split
camel-core into multiple parts (hadrian)</h4>

<p>(&#43;1: cmueller, hadrian)</p>

<p>There are multiple benefits and less constraints. A separate api jar would allow
the definition of a 'route container' which is currently one of the roles of the CamelContext.
This allows primarily alternative implementations of camel for constrained environments (such
as real time systems, for instance). Processors/Routes/Components written against the api
could be deployed on any camel implementation (as long as all necessary features are supported).</p>


<p>These should be structured in a way that these big building blocks do not have cyclic
dependencies. Any other cycles can be ignored in this step.</p>

<p>Allowed depdencies ( "-&gt;" means may use, may depend on):</p>

	<li>&#42; &#45;&gt; api</li>
	<li>end user config code &#45;&gt; builder</li>
	<li>builder &#45;&gt; impl</li>

<h4><a name="Camel3.0-Ideas-Definescopeandrulesforcamelcorepackages%28champion%3F%29"></a>Define
scope and rules for camel-core packages (<em>champion</em>?)</h4>

<p>In extension to the previous paragraph each camel package should have a clear scope
that defines what to put in the package and what not. There should be rules that define what
dependencies are allowed for classes in a package. The minimum goal is to guarantee that by
following the rules dependency cycles can not happen. Additionally the rules should minimize
dependencies between packages to achieve loose coupling between packages and high coherence
inside a package.</p>

<h4><a name="Camel3.0-Ideas-Moreflexibleroutesatruntime%28claus%29"></a>More
flexible routes at runtime (claus)</h4>
<p>(&#43;1: hadrian)</p>

<p>When routes is added in Camel 2.x architecture, global cross cutting concerns such
as error handlers, interceptors, onCompletion etc. is applied when the route is added. We
need to separate this and have those applied during routing. The <tt>Channel</tt>
needs to do this and therefore it must be more dynamic than its currently is. And we need
to enlist the various global cross cutting concerns by their xxxDefintions in the CamelContext,
so we can access them at any time. This allows end users also much more easily to add/remove
interceptors, error handlers and whatnot at runtime. And it makes it much easier to add routes
generated from JAXB or other sources, as we don't need to prepare or anyhow <em>mold</em>
the <tt>RouteDefinition</tt> given. See ticket <a href=""
class="external-link" rel="nofollow">CAMEL-3024</a> for some details.</p>

<h4><a name="Camel3.0-Ideas-Fixrouteswithmultipleinputs%28claus%29"></a>Fix
routes with multiple inputs (claus)</h4>

<p>The current implementation of routes with multiple inputs is to clone the route,
which means you essentially got 2&#43; routes if a route has multiple inputs. However
routes with multiple inputs is seldom used. The actual solution will depend on the api refactoring.</p>

<h4><a name="Camel3.0-Ideas-RouteinitializationlogicforJavaDSLandXMLDSLs%28claus%29"></a>Route
initialization logic for Java DSL and XML DSLs (claus)</h4>

<p>The Java DSL does its route initialization slightly a bit different than the XML
DSLs, due the nature of it, and the fact the fluent builders can do additional logic, which
the JAXB model of XML DSLs does not. We should align the initialization logic so Java DSL
and XML DSLs does the same thing. They setup the pure model at first. So the configure method
in the RouteBuilder should setup the model as the XML DSL would do. Then the prepare route
logic which follows could be the same in all cases. This would also allow us to ensure when
people use multiple RouteBuilder classes in Java DSL, then context scoped onException, interceptors
is applied for all RouteBuilders.</p>

<h4><a name="Camel3.0-Ideas-AddOnException%2CInterceptor%2Cetc.toJAXBmodelforaCamelContextDefinition%28claus%29"></a>Add
OnException, Interceptor, etc. to JAXB model for a CamelContextDefinition (claus)</h4>

<p>Configuring context scoped onException, interceptors etc. is woven into the RouteDefinition
as part of the route initialization logic. When we have a dynamic routing engine (see above)
that can at runtime support this without the need for woven into the routes. Then we should
also ensure the context scoped onException, interceptors etc. is available in a CamelContextDefinition.
This ensures the models is always 100% kept as it was provided, and we can fully export the
model to XML and other languages (having a supported render).</p>

<h4><a name="Camel3.0-Ideas-Tightenuproutedefinitions%28claus%29"></a>Tighten
up route definitions (claus)</h4>

<p>Currently cross cutting concerns such as error handlers, interceptors, onCompletion
etc. can be define anywhere in the route. We should tighten this up and only allow this to
be configured in the start of the route. This also ensures when end users use code assistance
in their route development, the IDE will not popup a big list which includes these cross cutting
concerns. See also next note. (ProcessorDefinition will therefore be trimmed)</p>

<h4><a name="Camel3.0-Ideas-MessageHistoryEIP%2FMessageStore%28ChristianOhr%29"></a>Message
History EIP/Message Store (Christian Ohr)</h4>

<p>(&#43;1: hadrian)<br/>
We should make this EIP easier to use for end users, but offering a better public API. And
also have a pluggable message store, with filters that can filter what should be stored. As
well pluggable marshallers so people can marshal data from Exchange into a format the message
store can store (BLOB, XML, JSon etc.).<br/>
A Message Store could provide transparent persistence to various EIP patterns (or used directly).
Implementations would handle the mapping to the underlying database or file system or NoSQL
or memory or whatever.<br/>
Message Store implementations are already there in various places, using different approaches,
like in Stream Caching (only file system), AggregationRepository or IdempotentRepository.
A Message Store is requested for in other places like Reliable stream resequencing (CAMEL-949)
or Persistent Dead Letter Queue (CAMEL-4575). And there might be other areas (seda, bam) that
might benefit as well.</p>

<h4><a name="Camel3.0-Ideas-Removenotusedcomponents"></a>Remove not used

<p>We should consider removing</p>
<ul class="alternate" type="square">
	<li>org.apache.camel.view from came-core</li>
	<li>dot maven generator</li>
	<li>... (there could be other stuff to remove)</li>

<p>The BAM has not changed in 5 years, and very seldom used by end users. And neither
has the functionality you need. There is much better solutions outside ASF Camel for a BAM
The DOT generator is not up to date and maintained. Also it requires binary generator to generate
a route diagram; we never managed to find a good java library for that.</p>

<h3><a name="Camel3.0-Ideas-Oldideas"></a>Old ideas</h3>
<p><em>To be better defined and moved to the section above or removed</em></p>

<h4><a name="Camel3.0-Ideas-Supportforasynchronoustransactions"></a>Support
for asynchronous transactions</h4>

<p>When using the asynchronous routing engine it would be desirable of transactional
context could be propagated to the new threads.<br/>
This requires the TX manager supports suspend/resume on the TX. G.Nodet have worked a bit
on this. See <a href="" class="external-link"
rel="nofollow">CAMEL-2902</a>. Also see <a href=""
class="external-link" rel="nofollow">CAMEL-2729</a>.</p>

<p>With the <a href="/confluence/display/CAMEL/Asynchronous+Routing+Engine" title="Asynchronous
Routing Engine">Asynchronous Routing Engine</a> it would be great if we could support
asynchronous transaction as well. See <a href=""
class="external-link" rel="nofollow">CAMEL-2729</a> and <a href=""
class="external-link" rel="nofollow">CAMEL-2902</a></p>

<h4><a name="Camel3.0-Ideas-Streamcaching"></a>Stream caching</h4>

<p>We could add support for using <a href="/confluence/display/CAMEL/HawtDB" title="HawtDB">HawtDB</a>
as the persistent store for streams which overflow to disk store.</p>

<h4><a name="Camel3.0-Ideas-EIP"></a>EIP</h4>

<p>The <a href="/confluence/display/CAMEL/Resequencer" title="Resequencer">Resequencer</a>
EIP currently doesn't support persistence, we could introduce this and let it leverage <a
href="/confluence/display/CAMEL/HawtDB" title="HawtDB">HawtDB</a> such as we did
with the <a href="/confluence/display/CAMEL/Aggregator2" title="Aggregator2">Aggregator2</a>

<h4><a name="Camel3.0-Ideas-ScheduleinDSL"></a>Schedule in DSL</h4>

<p>We could consider adding DSL syntax sugar for scheduling routes. For example currently
you have to use <a href="/confluence/display/CAMEL/Quartz" title="Quartz">Quartz</a>
or a <tt>ScheduledPollingConsumer</tt> which has the <tt>delay</tt>
option. We could add DSL which has something like:</p>

<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
schedule().every(5).minute().pollFrom(<span class="code-quote">"xxx"</span>).to(<span

<p>The hard part is to come up with a good DSL syntax. We can look at <a href="/confluence/display/CAMEL/BAM"
title="BAM">BAM</a> and see what we got there as well.</p>

<p>The DSL should support both cron and non cron based, eg <a href="/confluence/display/CAMEL/Quartz"
title="Quartz">Quartz</a>, <a href="/confluence/display/CAMEL/Spring" title="Spring">Spring</a>
(spring 3 has cron) and regular JDK timers.</p>

<h4><a name="Camel3.0-Ideas-AdvancedScalasupport"></a>Advanced Scala support</h4>

<p>-1: claus</p>

<p>A proposal for advanced Scala support is <a href=""
class="external-link" rel="nofollow">scalaz-camel</a> which is an alternative to
Camel's existing Scala DSL.</p>

<h4><a name="Camel3.0-Ideas-MoreEIPsas@annotations"></a>More EIPs as @annotations</h4>

<p>Currently its only the <a href="/confluence/display/CAMEL/Routing+Slip" title="Routing
Slip">Routing Slip</a>, <a href="/confluence/display/CAMEL/Recipient+List" title="Recipient
List">Recipient List</a> and <a href="/confluence/display/CAMEL/Dynamic+Router"
title="Dynamic Router">Dynamic Router</a> which are avail as @annotation as well.
We could add more <a href="/confluence/display/CAMEL/EIP" title="EIP">EIP</a>s
as annotations such as <a href="/confluence/display/CAMEL/Splitter" title="Splitter">Splitter</a>.<br/>
And also maybe annotations for <tt>AggregationStrategy</tt> to make this less
Camel API dependent, so you can use a plain POJO for that.</p>

<h4><a name="Camel3.0-Ideas-Unifiedstatistics"></a>Unified statistics</h4>

<p>-1: claus. We keep the JMX API and ppl can use jolokia for REST over JMX etc.</p>

<p>Currently the performance statistics is only avail when using JMX. We should allow
those stats to be enabled regardless if JMX is enabled or not. Then we can use those stats
from the web console. This also allows to expose those stats in the cloud where JMX is often
not possible to be used.</p>

<p>In the <tt>camel-jpa</tt> component we could offer JPA annotated entities
with the performance status (just as we do for the tracer). Then end user can more easily
use that if they want the stats to be persisted in a database using <a href="/confluence/display/CAMEL/JPA"

<h4><a name="Camel3.0-Ideas-SEDA%2FVMcomponentstoleverageasyncroutingengine"></a>SEDA/VM
components to leverage async routing engine</h4>

<p>This allows to use non blocking request-reply over <a href="/confluence/display/CAMEL/SEDA"
title="SEDA">SEDA</a> and <a href="/confluence/display/CAMEL/VM" title="VM">VM</a>.
The reason why we havent converted in 2.4 is it causes a bigger API breakage.</p>

<h4><a name="Camel3.0-Ideas-camelosgitest"></a>camel-osgi-test</h4>

<p>When testing your Camel apps with OSGi you may use PaxExam for that. We should create
a test kit for osgi, like we have camel-test for regular junit testing. The test kit should
make it easy for end users to have their apps tested with OSGi. We already have pieces in
the <tt>tests/camel-itest-osgi</tt>. We just need to clean and shape it up so
its ready for end users as well. And of course add documentation as well.<br/>
And then we should use it in <tt>camel-itest-osgi</tt> of course. See <a href=""
class="external-link" rel="nofollow">CAMEL-3777</a>.</p>

<h4><a name="Camel3.0-Ideas-REST"></a>REST</h4>

<p>We already have REST support with <a href="/confluence/display/CAMEL/CXFRS" title="CXFRS">CXFRS</a>
and <a href="/confluence/display/CAMEL/Restlet" title="Restlet">Restlet</a> but
it can be better. We should make sure those components is dead easy to use and you can invoke
REST services in one line of code etc. And we should make more examples and tidy up the <a
href="/confluence/display/CAMEL/CXFRS" title="CXFRS">CXFRS</a> documentation.</p>

<h4><a name="Camel3.0-Ideas-Moreloadtests%28cmueller%29"></a>More load tests

<p>More load tests for frequently used Camel components (jetty, jms ...) and camel-core.
If we have an defined runtime environment which is in general accessable/available for all
users (like a public image on Amazon EC2) and a data store for the performance numbers, than
we could easily collect and share the numbers.</p>
	<li>Ensure correct behavior under load</li>
	<li>Source for performance numbers (throughput etc).</li>
	<li>Detection of memory leaks</li>
	<li>Detection of performance decreases after refactorings</li>

<h4><a name="Camel3.0-Ideas-OSGienhancements"></a>OSGi enhancements</h4>

	<li>create a single type converter registry available as OSGi service to all bundles
serving as a single registry to lookup converters or to add/remove converters from custom

<h4><a name="Camel3.0-Ideas-Easiercommit%2Frollbackforcomponentdevelopers"></a>Easier
commit/rollback for component developers</h4>

<p>Maybe expose some interface having commit / rollback methods to make it easier for
component developers to implement custom logic. Currently they may not know about <a href="/confluence/display/CAMEL/OnCompletion"
title="OnCompletion">OnCompletion</a> and how to use <tt>UnitOfWork</tt>
on Exchange to do this today.</p>

<h4><a name="Camel3.0-Ideas-Unifyuri%2Fref%28cmueller%29"></a>Unify uri/ref
<p>+1: claus, cmueller</p>

<p>Instead of having both uri and ref for endpoints, we should unify this and only use
uri. If people want to use ref, then they can do that using "ref:xx" as an uri. This would
simplify code as we dont have to check for either one.</p>

<h4><a name="Camel3.0-Ideas-removethexxxRefoptions%28cmueller%29"></a>remove
the xxxRef options (cmueller)</h4>
<p>+1: claus, cmueller</p>

<p>In favor of the xxx option, we should remove the duplicated xxxRef options from our
components. This make our code and documentation more lean. We should deprecate it as soon
as possible and remove it in Camel 3.0.</p>

<h4><a name="Camel3.0-Ideas-RefactorUnitOfWork"></a>Refactor UnitOfWork</h4>

<p>The implementation of DefaultUnitOfWork seems to have transformed itself into a to
broad concern where unit of work is doing a bit more work than the transactional aspect that
ties to its name.<br/>
Maybe this implementation should be named ExchangeContext and we can introduce a simpler UnitOfWork
concept. This would also allow us to refactor the SubUnitOfWork into a general parent/child
unit of work concept.<br/>
However this requires API changes and thus is best kept for Camel 3.0</p>

<p>This is also needed by the refactor of the Message History EIP.</p>

<h4><a name="Camel3.0-Ideas-ImprovementstoThreadPoolProfileforthreadmanagement"></a>Improvements
to ThreadPoolProfile for thread management</h4>

<p>We could move <tt>ThreadPoolProfile</tt> from <tt>org.apache.camel.spi</tt>
to <tt>org.apache.camel</tt> and have it in the root package.</p>

<h4><a name="Camel3.0-Ideas-TightenuponException"></a>Tighten up onException</h4>

<p>We should consider tighten up the onException DSL a bit, for example to make it more
clear that if you have processing steps in there, they only occur after the message has been
exhausted. So maybe we need to have a onExhausted to make this clear</p>

<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
  .onExhausted().handled(<span class="code-keyword">true</span>).to(<span class="code-quote">"log:ignoreMe"</span>);

<p>So in this example its only after the 3 failed redeliveries, then its exhausted,
and then we handle the message and route it to a log where we ignore the message.</p>

<p>Also currently you can do this:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
  .handled(<span class="code-keyword">true</span>);

<p>Which will handle the message and after this let the error handler deal with the
message. See <a href="" class="external-link"
We should in this case use a NoopProcessor so the message is handled, but the regular error
handler does not react. Then its consistent.<br/>
However we have not changed this in Camel 2.x to keep backwards compatibility.</p>

<p>Likewise there has been ideas to move onRedeliverRef to &lt;redeliveryPolicy&gt;
as currently its to be configured outside the policy.<br/>
It may make more sense to move onRedeliverRef to the policy to keep it together.</p>

<h4><a name="Camel3.0-Ideas-AddcompositeEIPtocomposeanumberofEIPsintoasingleEIP"></a>Add
composite EIP to compose a number of EIPs into a single EIP</h4>

<p>It would be nice if we have a DSL to compose a number of child EIPs into a single
EIP itself. eg a bit like &lt;pipeline&gt;. This would allow people to make it easier
to group together a number of EIPs into a single "unit". This allows for example the error
handler to redeliver to the composite EIP instead of at the point of failure within the group.
There are use-cases where people want to do that. And today the solution is to split this
into a new route, which you then disable error handler by setting it to no error handler.
And then call the route using the direct endpoint. So instead if you could do &lt;composite&gt;
... stuff goes here &lt;/composite&gt; then that would be neater, as you dont have
to split into multiple routes. I think there is an old JIRA ticket created about this a long
time ago.</p>

        <div id="commentsSection" class="wiki-content pageSection">
        <div style="float: right;">
            <a href=""
class="grey">Change Notification Preferences</a>
        <a href="">View
        <a href="">View
        <a href=";showCommentArea=true#addcomment">Add

View raw message