camel-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From conflue...@apache.org
Subject [CONF] Apache Camel > Zookeeper
Date Wed, 24 Aug 2011 00:19:00 GMT
<html>
<head>
    <base href="https://cwiki.apache.org/confluence">
            <link rel="stylesheet" href="/confluence/s/2042/9/1/_/styles/combined.css?spaceKey=CAMEL&amp;forWysiwyg=true"
type="text/css">
    </head>
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
    <h2><a href="https://cwiki.apache.org/confluence/display/CAMEL/Zookeeper">Zookeeper</a></h2>
    <h4>Page <b>edited</b> by             <a href="https://cwiki.apache.org/confluence/display/~sgargan">Stephen
Gargan</a>
    </h4>
        <br/>
                         <h4>Changes (8)</h4>
                                 
    
<div id="page-diffs">
                    <table class="diff" cellpadding="0" cellspacing="0">
    
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >{code} <br> <br></td></tr>
            <tr><td class="diff-changed-lines" >ZooKeeper nodes can have different
types; they can be &#39;Ephemeral&#39; or &#39;Persistent&#39; and &#39;Sequenced&#39;
or &#39;Unsequenced&#39;. For further information of each type you can check <span
class="diff-changed-words">[here|http://zookeeper.apache.org/doc/trunk/zookeeperProgrammers.html#Ephemeral+Nodes]<span
class="diff-added-chars"style="background-color: #dfd;">. By default endpoints will create
unsequenced, ephemeral nodes, but the type can be easily manipulated via a uri config parameter
or via a special message header. The values expected for the create mode are simply the names
from the CreateMode enumeration</span></span> <br></td></tr>
            <tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">By
default endpoints will create unsequenced, ephemeral nodes, but the type can be easily manipulated
via a uri config parameter or via a special message header. The values expected for the create
mode are simply the names from the CreateMode enumeration <br></td></tr>
            <tr><td class="diff-unchanged" > <br></td></tr>
            <tr><td class="diff-changed-lines" ><span class="diff-deleted-words"style="color:#999;background-color:#fdd;text-decoration:line-through;">#</span>
<span class="diff-added-words"style="background-color: #dfd;">-</span> PERSISTENT
<br></td></tr>
            <tr><td class="diff-changed-lines" ><span class="diff-deleted-words"style="color:#999;background-color:#fdd;text-decoration:line-through;">#</span>
<span class="diff-added-words"style="background-color: #dfd;">-</span> PERSISTENT_SEQUENTIAL
<br></td></tr>
            <tr><td class="diff-changed-lines" ><span class="diff-deleted-words"style="color:#999;background-color:#fdd;text-decoration:line-through;">#</span>
<span class="diff-added-words"style="background-color: #dfd;">-</span> EPHEMERAL
<br></td></tr>
            <tr><td class="diff-changed-lines" ><span class="diff-deleted-words"style="color:#999;background-color:#fdd;text-decoration:line-through;">#</span>
<span class="diff-added-words"style="background-color: #dfd;">-</span> EPHEMERAL_SEQUENTIAL
<br></td></tr>
            <tr><td class="diff-unchanged" > <br>For example to create a
persistent znode via the URI config <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >ZooKeeper allows for very simple and
effective leader election out of the box; This component exploits this election capability
in a RoutePoilcy to control when and how routes are enabled. This policy would typically be
used in fail-over scenarios, to control identical instances of a route across a cluster of
Camel based servers. A very common scenarion is a simple &#39;Master-Slave&#39; setup
where there are multiple instances of a route distributed across a cluster but only one of
them, that of the master, should be running at a time. If the master fails, a new master should
be elected from the available slaves and the route in this new master should be started. <br>
<br></td></tr>
            <tr><td class="diff-changed-lines" >The policy uses a common znode
path across all instances of the RoutePolicy that will be involved in the election. Each policy
writes its id into this node and zookeeper will order the writes in the order it received
them. <span class="diff-added-words"style="background-color: #dfd;">The policy then
reads the listing of the node to see what postion of its id; this postion is used to determine
if the route should be started or not. The policy is configured at starup with the number
of route instances that should be started across the cluster and if its position in the list
is less than this value then its route will be started. For a Master-slave scenario, the route
is configured with 1 route instance and only the first entry in the listing will start its
route. All policies watch for updates to the listing and if the listing changes they recaclulate
if their route should be started. For more info on Zookeeper&#39;s Leader election capability
[see|http://zookeeper.apache.org/doc/trunk/recipes.html#sc_leaderElection]</span> <br></td></tr>
            <tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">The
policy then reads the listing of the node to see what postion of its id; this postion is used
to determine if the route should be started or not. The policy is configured at starup with
the number of route instances that should be started across the cluster and if its position
in the list is less than this value then its route will be started. For a Master-slave scenario,
the route is configured with 1 route instance and only the first entry in the listing will
start its route. All policies watch for updates to the listing and if the listing changes
they recaclulate if their route should be started. For more info on Zookeeper&#39;s Leader
election capability [see|http://zookeeper.apache.org/doc/trunk/recipes.html#sc_leaderElection]
 <br></td></tr>
            <tr><td class="diff-unchanged" > <br>The following example uses
the node &#39;/someapplication/somepolicy&#39; for the election and is set up to start
only the top &#39;1&#39; entries in the node listing i.e. elect a master <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
    
            </table>
    </div>                            <h4>Full Content</h4>
                    <div class="notificationGreySide">
        <h2><a name="Zookeeper-ZooKeeperComponent"></a>ZooKeeper Component</h2>
<p><b>Available as of Camel 2.9</b></p>

<p>The ZooKeeper component to allow interaction with a ZooKeeper cluster (<a href="http://hadoop.apache.org/zookeeper/"
class="external-link" rel="nofollow">http://hadoop.apache.org/zookeeper/</a>) and
it exposes the following features to Camel.</p>

<ol>
	<li>Creation of nodes in any of the ZooKeeper create modes.</li>
	<li>Get and Set the data contents of arbitrary cluster nodes.</li>
	<li>Create and retrieve the list the child nodes attached to a particular node.</li>
	<li>A Distributed RoutePoilcy that leverages a Leader election coordinated by ZK to
determine if exchanges should get processed.</li>
</ol>


<p>Maven users will need to add the following dependency to their <tt>pom.xml</tt>
for this component:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-xml">
<span class="code-tag">&lt;dependency&gt;</span>
    <span class="code-tag">&lt;groupId&gt;</span>org.apache.camel<span
class="code-tag">&lt;/groupId&gt;</span>
    <span class="code-tag">&lt;artifactId&gt;</span>camel-zookeeper<span
class="code-tag">&lt;/artifactId&gt;</span>
    <span class="code-tag">&lt;version&gt;</span>x.x.x<span class="code-tag">&lt;/version&gt;</span>
    <span class="code-tag"><span class="code-comment">&lt;!-- use the same
version as your Camel core version --&gt;</span></span>
<span class="code-tag">&lt;/dependency&gt;</span>
</pre>
</div></div>

<h3><a name="Zookeeper-URIformat"></a>URI format</h3>

<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
zookeeper:<span class="code-comment">//zookeeper-server[:port][/path][?options]</span>
</pre>
</div></div>

<p>The path from the uri specifies the node in the ZooKeeper server (aka znode) that
will be the target of the endpoint.</p>

<h3><a name="Zookeeper-Options"></a>Options</h3>
<div class="confluenceTableSmall"><div class='table-wrap'>
<table class='confluenceTable'><tbody>
<tr>
<th class='confluenceTh'> Name </th>
<th class='confluenceTh'> Default Value </th>
<th class='confluenceTh'> Description </th>
</tr>
<tr>
<td class='confluenceTd'> <tt>sessionId</tt> </td>
<td class='confluenceTd'> <tt>null</tt> </td>
<td class='confluenceTd'> The session id used to identify a connection to the cluster
</td>
</tr>
<tr>
<td class='confluenceTd'> <tt>password</tt> </td>
<td class='confluenceTd'> <tt>NULL</tt> </td>
<td class='confluenceTd'> The password to use when making a connection </td>
</tr>
<tr>
<td class='confluenceTd'> <tt>awaitCreation</tt> </td>
<td class='confluenceTd'> <tt>true</tt> </td>
<td class='confluenceTd'> should the endpoint await the creation of a node that does
not yet exist.</td>
</tr>
<tr>
<td class='confluenceTd'> <tt>listChildren</tt> </td>
<td class='confluenceTd'> <tt>false</tt> </td>
<td class='confluenceTd'> whether the children of the node should be listed</td>
</tr>
<tr>
<td class='confluenceTd'> <tt>repeat</tt> </td>
<td class='confluenceTd'> <tt>false</tt> </td>
<td class='confluenceTd'> should changes to the znode be 'watched' and repeatedly processed.
</td>
</tr>
<tr>
<td class='confluenceTd'> <tt>backoff</tt> </td>
<td class='confluenceTd'> <tt>5000</tt> </td>
<td class='confluenceTd'> the time interval to backoff for after an error before retrying.
 </td>
</tr>
<tr>
<td class='confluenceTd'> <tt>timeout</tt> </td>
<td class='confluenceTd'> <tt>5000</tt> </td>
<td class='confluenceTd'> the time interval to wait on connection before timing out.
</td>
</tr>
<tr>
<td class='confluenceTd'> <tt>create</tt> </td>
<td class='confluenceTd'> <tt>false</tt> </td>
<td class='confluenceTd'> should the endpoint create the node if it does not currently
exist. </td>
</tr>
<tr>
<td class='confluenceTd'> <tt>createMode</tt> </td>
<td class='confluenceTd'> <tt>EPHEMERAL</tt> </td>
<td class='confluenceTd'> the create mode that should be used for the newly created
node (see below). </td>
</tr>
</tbody></table>
</div>
</div>

<h3><a name="Zookeeper-Usecases"></a>Use cases</h3>

<h4><a name="Zookeeper-Readingfromaznode."></a>Reading from a znode.</h4>

<p>The following snippet will read the data from the znode '/somepath/somenode/' provided
that it already exists. The data retrieved will<br/>
be placed into an exchange and passed onto the rest of the route.</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
from(<span class="code-quote">"direct:write-to-znode"</span>).to(<span class="code-quote">"zoo:<span
class="code-comment">//localhost:39913/somepath/somenode"</span>);</span>
</pre>
</div></div>

<p>if the node does not yet exist then a flag can be supplied to have the endpoint await
its creation</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
from(<span class="code-quote">"direct:create-and-write-to-znode"</span>).to(<span
class="code-quote">"zoo:<span class="code-comment">//localhost:39913/somepath/somenode?awaitCreation=<span
class="code-keyword">true</span>"</span>);</span>
</pre>
</div></div>

<h4><a name="Zookeeper-Writingtoaznode."></a>Writing to a znode.</h4>

<p>the following snippet will write the payload of the exchange into the znode at '/somepath/somenode/'
provided that it already exists </p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
from(<span class="code-quote">"direct:write-to-znode"</span>).to(<span class="code-quote">"zoo:<span
class="code-comment">//localhost:39913/somepath/somenode"</span>);</span>
</pre>
</div></div>

<p>For flexibility, the endpoint allows the target znode to be specified dynamically
as a message header. If a header keyed by the string 'CamelZooKeeperNode' is present then
the value of the header will be used as the path to the znode on the server. For instance
using the same route definition above, the following code snippet will write the data not
to '/somepath/somenode' but to the path from the header '/somepath/someothernode'</p>

<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
Exchange e = createExchangeWithBody(testPayload);
template.sendBodyAndHeader(<span class="code-quote">"direct:write-to-znode"</span>,
e, <span class="code-quote">"CamelZooKeeperNode"</span>, <span class="code-quote">"/somepath/someothernode"</span>);
</pre>
</div></div>

<p>To also create the node if it does not exist the 'create' option should be used.
 </p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
from(<span class="code-quote">"direct:create-and-write-to-znode"</span>).to(<span
class="code-quote">"zoo:<span class="code-comment">//localhost:39913/somepath/somenode?create=<span
class="code-keyword">true</span>"</span>);</span>
</pre>
</div></div>

<p>ZooKeeper nodes can have different types; they can be 'Ephemeral' or 'Persistent'
and 'Sequenced' or 'Unsequenced'. For further information of each type you can check <a
href="http://zookeeper.apache.org/doc/trunk/zookeeperProgrammers.html#Ephemeral+Nodes" class="external-link"
rel="nofollow">here</a>. By default endpoints will create unsequenced, ephemeral
nodes, but the type can be easily manipulated via a uri config parameter or via a special
message header. The values expected for the create mode are simply the names from the CreateMode
enumeration</p>

<ul class="alternate" type="square">
	<li>PERSISTENT</li>
	<li>PERSISTENT_SEQUENTIAL</li>
	<li>EPHEMERAL</li>
	<li>EPHEMERAL_SEQUENTIAL</li>
</ul>


<p>For example to create a persistent znode via the URI config</p>

<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
from(<span class="code-quote">"direct:create-and-write-to-persistent-znode"</span>).to(<span
class="code-quote">"zoo:<span class="code-comment">//localhost:39913/somepath/somenode?create=<span
class="code-keyword">true</span>&amp;createMode=PERSISTENT"</span>);</span>
</pre>
</div></div>

<p>or using the header 'CamelZookeeperCreateMode'</p>

<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
Exchange e = createExchangeWithBody(testPayload);
template.sendBodyAndHeader(<span class="code-quote">"direct:create-and-write-to-persistent-znode"</span>,
e, <span class="code-quote">"CamelZooKeeperCreateMode"</span>, <span class="code-quote">"PERSISTENT"</span>);
</pre>
</div></div>


<h3><a name="Zookeeper-ZooKeeperenabledRoutepolicy."></a>ZooKeeper enabled
Route policy.</h3>

<p>ZooKeeper allows for very simple and effective leader election out of the box; This
component exploits this election capability in a RoutePoilcy to control when and how routes
are enabled. This policy would typically be used in fail-over scenarios, to control identical
instances of a route across a cluster of Camel based servers. A very common scenarion is a
simple 'Master-Slave' setup where there are multiple instances of a route distributed across
a cluster but only one of them, that of the master, should be running at a time. If the master
fails, a new master should be elected from the available slaves and the route in this new
master should be started.</p>

<p>The policy uses a common znode path across all instances of the RoutePolicy that
will be involved in the election. Each policy writes its id into this node and zookeeper will
order the writes in the order it received them. The policy then reads the listing of the node
to see what postion of its id; this postion is used to determine if the route should be started
or not. The policy is configured at starup with the number of route instances that should
be started across the cluster and if its position in the list is less than this value then
its route will be started. For a Master-slave scenario, the route is configured with 1 route
instance and only the first entry in the listing will start its route. All policies watch
for updates to the listing and if the listing changes they recaclulate if their route should
be started. For more info on Zookeeper's Leader election capability <a href="http://zookeeper.apache.org/doc/trunk/recipes.html#sc_leaderElection"
class="external-link" rel="nofollow">see</a> </p>

<p>The following example uses the node '/someapplication/somepolicy' for the election
and is set up to start only the top '1' entries in the node listing i.e. elect a master</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
ZooKeeperRoutePolicy policy = <span class="code-keyword">new</span> ZooKeeperRoutePolicy(<span
class="code-quote">"zoo:localhost:39913/someapp/somepolicy"</span>, 1);
from(<span class="code-quote">"direct:policy-controlled"</span>).routePolicy(policy).to(<span
class="code-quote">"mock:controlled"</span>);
</pre>
</div></div>        
    </div>
        <div id="commentsSection" class="wiki-content pageSection">
        <div style="float: right;">
            <a href="https://cwiki.apache.org/confluence/users/viewnotifications.action"
class="grey">Change Notification Preferences</a>
        </div>
        <a href="https://cwiki.apache.org/confluence/display/CAMEL/Zookeeper">View Online</a>
        |
        <a href="https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=27825114&revisedVersion=2&originalVersion=1">View
Changes</a>
                |
        <a href="https://cwiki.apache.org/confluence/display/CAMEL/Zookeeper?showComments=true&amp;showCommentArea=true#addcomment">Add
Comment</a>
            </div>
</div>
</div>
</div>
</div>
</body>
</html>

Mime
View raw message