geronimo-scm mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From conflue...@apache.org
Subject [CONF] Apache Geronimo v3.0 > Plugin based Farming
Date Tue, 25 Jan 2011 09:20:00 GMT
<html>
<head>
    <base href="https://cwiki.apache.org/confluence">
            <link rel="stylesheet" href="/confluence/s/2036/9/4/_/styles/combined.css?spaceKey=GMOxDOC30&amp;forWysiwyg=true"
type="text/css">
    </head>
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
    <h2><a href="https://cwiki.apache.org/confluence/display/GMOxDOC30/Plugin+based+Farming">Plugin
based Farming</a></h2>
    <h4>Page <b>edited</b> by             <a href="https://cwiki.apache.org/confluence/display/~maojia508">maojia</a>
    </h4>
        <br/>
                         <h4>Changes (2)</h4>
                                 
    
<div id="page-diffs">
                    <table class="diff" cellpadding="0" cellspacing="0">
    
            <tr><td class="diff-unchanged" >{scrollbar} <br> <br></td></tr>
            <tr><td class="diff-changed-lines" >{excerpt}This farming system uses
plugins <span class="diff-deleted-words"style="color:#999;background-color:#fdd;text-decoration:line-through;">directly
and</span> <span class="diff-added-words"style="background-color: #dfd;">directly.
Therefore, it</span> is <span class="diff-deleted-words"style="color:#999;background-color:#fdd;text-decoration:line-through;">thus</span>
decoupled from the deployment system and allows running the plugin packaging (or &quot;deployment&quot;)
and farm administration on different servers or machines.{excerpt}  The current implementation
uses multicast for node discovery.  Farming information is stored in a database using jps.
 The data model allows a single administration server to manage: <br></td></tr>
            <tr><td class="diff-unchanged" >* multiple farms (currently sometimes
termed _clusters_) <br>* multiple lists of plugins per farm (in an mXn relationship)
<br>* multiple plugins per plugin list <br> <br></td></tr>
            <tr><td class="diff-changed-lines" >When a node starts <span class="diff-changed-words">up<span
class="diff-added-chars"style="background-color: #dfd;">,</span></span> it
broadcasts a service advertisement including an optional node name (defaulting to host:jndi-port)
and a farm name. When the administration server detects this as <span class="diff-changed-words">new<span
class="diff-added-chars"style="background-color: #dfd;">,</span></span> it
instructs the new node to install all the plugin lists associated with that farm.  If the
plugins are already installed, this is a no-op; otherwise the plugins are downloaded and installed
from the plugin repository specified in the plugin list. <br></td></tr>
            <tr><td class="diff-unchanged" > <br>Administrative operations
include: <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
    
            </table>
    </div>                            <h4>Full Content</h4>
                    <div class="notificationGreySide">
        <style type='text/css'>/*<![CDATA[*/
table.ScrollbarTable  {border: none;padding: 3px;width: 100%;padding: 3px;margin: 0px;background-color:
#f0f0f0}
table.ScrollbarTable td.ScrollbarPrevIcon {text-align: center;width: 16px;border: none;}
table.ScrollbarTable td.ScrollbarPrevName {text-align: left;border: none;}
table.ScrollbarTable td.ScrollbarParent {text-align: center;border: none;}
table.ScrollbarTable td.ScrollbarNextName {text-align: right;border: none;}
table.ScrollbarTable td.ScrollbarNextIcon {text-align: center;width: 16px;border: none;}

/*]]>*/</style><div class="Scrollbar"><table class='ScrollbarTable'><tr><td
class='ScrollbarPrevIcon'><a href="/confluence/display/GMOxDOC30/JMS+clustering+in+Geronimo"><img
border='0' align='middle' src='/confluence/images/icons/back_16.gif' width='16' height='16'></a></td><td
width='33%' class='ScrollbarPrevName'><a href="/confluence/display/GMOxDOC30/JMS+clustering+in+Geronimo">JMS
clustering in Geronimo</a>&nbsp;</td><td width='33%' class='ScrollbarParent'><sup><a
href="/confluence/display/GMOxDOC30/Clustering+and+farming"><img border='0' align='middle'
src='/confluence/images/icons/up_16.gif' width='8' height='8'></a></sup><a
href="/confluence/display/GMOxDOC30/Clustering+and+farming">Clustering and farming</a></td><td
width='33%' class='ScrollbarNextName'>&nbsp;<a href="/confluence/display/GMOxDOC30/Tomcat+Native+Clustering">Tomcat
Native Clustering</a></td><td class='ScrollbarNextIcon'><a href="/confluence/display/GMOxDOC30/Tomcat+Native+Clustering"><img
border='0' align='middle' src='/confluence/images/icons/forwd_16.gif' width='16' height='16'></a></td></tr></table></div>

<p>This farming system uses plugins directly. Therefore, it is decoupled from the deployment
system and allows running the plugin packaging (or "deployment") and farm administration on
different servers or machines.  The current implementation uses multicast for node discovery.
 Farming information is stored in a database using jps.  The data model allows a single administration
server to manage:</p>
<ul>
	<li>multiple farms (currently sometimes termed <em>clusters</em>)</li>
	<li>multiple lists of plugins per farm (in an mXn relationship)</li>
	<li>multiple plugins per plugin list</li>
</ul>


<p>When a node starts up, it broadcasts a service advertisement including an optional
node name (defaulting to host:jndi-port) and a farm name. When the administration server detects
this as new, it instructs the new node to install all the plugin lists associated with that
farm.  If the plugins are already installed, this is a no-op; otherwise the plugins are downloaded
and installed from the plugin repository specified in the plugin list.</p>

<p>Administrative operations include:</p>
<ul>
	<li>Add a plugin list to a farm</li>
	<li>Add a plugin to a plugin list</li>
	<li>Add a plugin to a plugin list and the list to a farm</li>
	<li>Remove a plugin list from a farm</li>
	<li>Remove a plugin from a plugin list</li>
</ul>


<p>(I haven't been able to think of a plausible meaning for "remove a plugin from a
plugin list from a farm")</p>

<p>These operations are available through the GShell <b>cluster/deploy</b>
command.</p>

<h2><a name="PluginbasedFarming-Missingfeatures"></a>Missing features</h2>
<ul>
	<li>Admin console support for the administrative operations</li>
	<li>Better support for setting up and starting multiple servers on one geronimo installation.</li>
</ul>


<h2><a name="PluginbasedFarming-ArchitecturalConsiderations"></a>Architectural
Considerations</h2>
<p>Two separate sets of functionality are required for plugin based farms:</p>
<ol>
	<li>A plugin farm controller that tracks the farms node, plugin lists, and plugins
and issues instructions to the nodes. The minimum configuration for this is the framework
server with the o.a.g.configs/plugin-farm//car plugin installed.</li>
	<li>A geronimo plugin repository accessible to all farm nodes. This can be a geronimo
server running the geronimo-as-maven servlet, the local file system maven repository, a file
based repository served by httpd, or a maven repository manager such as nexus. If a geronimo
server is used it can be the same server as the plugin farm controller or a separate server.</li>
</ol>


<h3><a name="PluginbasedFarming-Possiblepluginorientedproductionworkflowusingmaven"></a>Possible
plugin-oriented production workflow using maven</h3>
<p>This scenario assumes use of maven as a build tool and separate development, quality
assurance (QA), and production groups. It needs two maven repositories, for development and
production artifacts.  These can be simple file/httpd based repositories or something like
nexus. Code transfer between groups is in the form of geronimo plugins packaged by the geronimo
car-maven-plugin.</p>
<ol>
	<li>Development group deploys plugins into the development repository.</li>
	<li>QA group tests these candidate plugins and upon acceptance deploys them to the
production repository.</li>
	<li>Production group distributes plugins to appropriate farms.</li>
</ol>



<h2><a name="PluginbasedFarming-Supportingpluginsandassemblies"></a>Supporting
plugins and assemblies</h2>
<ul>
	<li>o.a.g.configs/plugin-farm-member//car (plugins/clustering/plugin-farm-member) plugin
turns a server into a plugin farm node. You can configure the multicast address and port,
cluster name, and node name in config-substitutions.properties.  It is probably most convenient
to not configure node name and rely on the default host:jndi-port.</li>
	<li>o.a.g.assemblies/geronimo-plugin-farm-node (plugins/clustering/geronimo-plugin-farm-node)
extends the framework assembly with the plugin-farm-member plugin.  This is all you need to
start up a farm node.</li>
	<li>o.a.g.configs/plugin-farm//car (plugins/clustering/plugin-farm) is the administration
plugin for plugin-based farms.  Currently it's only accessible through the gshell cluster/deploy
command.</li>
	<li>o.a.g.configs/plugin-farm-datasource//car (plugins/clustering/plugin-farm-datasource)
is a derby datasource used by the plugin farm. It is intended to be replaced in production
with a remote datasource. As always, the replacement plugin should use an &lt;artifact-alias&gt;
element to redirect dependencies on plugin-farm-datasource to itself.</li>
</ul>


<p>In my sandbox until we find a better location or solution, as examples</p>
<ul>
	<li>o.a.g.configs/controller-as-repo//car (sandbox/djencks/assemblies/controller-as-repo)
is a non-classloader plugin that sets the default plugin repository to be the admin server.
 This currently only works if both the admin server and all the farm nodes are running on
the same machine &#8211; obviously only a toy example scenario.</li>
	<li>o.a.g.assemblies/geronimo-tomcat-farm-controller//tar.gz|zip (sandbox/assemblies/geronimo-tomcat-farm-controller)
is a server assembly that extends the tomcat javaee5 server with the plugin farm controller</li>
</ul>


<h2><a name="PluginbasedFarming-GShellcluster%2Fdeploycommandsyntax"></a>GShell
cluster/deploy command syntax</h2>
<p>The GShell <b>cluster/deploy</b> command takes one argument, the action,
which must be <b>add</b> or <b>remove</b>, and options for the farm,
plugin list, and plugin.</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
-f (--farm) farm name
-l (--pluginlist) plugin list name
-a (--pluginartifact) plugin artifact name
</pre>
</div></div>
<p>In addition there are some inherited options for connecting to the server:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
-s (--hostname, --server) server name or IP address
-p (--port) server port
-u (--user) user name
-w (--password) password
--secure (use secure communication to server)
</pre>
</div></div>

<h2><a name="PluginbasedFarming-Examplewalkthrough"></a>Example walkthrough</h2>
<p>This example demonstrates the plugin farm on a single server, using nodes that all
share the same geronimo installation, and using the admin server as farm controller, application
deployment server, and plugin repository.  We'll install one of the sample applications on
the admin server and deploy it to the farm.  I've only tested this with all components built
locally on one machine, but some such as the geronimo-plugin-farm-node should be available
from the apache snapshot repository.</p>

<h3><a name="PluginbasedFarming-Farmnodesetup"></a>Farm node setup</h3>
<ol>
	<li>Obtain the geronimo-plugin-farm-node assembly, such as by building plugins/clustering.</li>
	<li>Unpack it and copy the var directory to multiple servers, say server1 through server5.
On a unix like system this can be done with a script like
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
#!/bin/bash

tar xzf geronimo-plugin-farm-node-2.2-SNAPSHOT-bin.tar.gz
cd geronimo-plugin-farm-node-2.2-SNAPSHOT

<span class="code-keyword">for</span> s in {1..5}
<span class="code-keyword">do</span> 
  sn=<span class="code-quote">"server${s}"</span>
  mkdir ${sn}
  cp -r <span class="code-keyword">var</span> ${sn}/
#uncomment to actually start the servers in the background
# ./bin/gsh geronimo/start-server -G server.name=${sn} -G config.substitution.PortOffset=${s}0
&gt;${sn}.log &amp;
done

cd ..
</pre>
</div></div>
<p>Here's a fancier script that can both start and stop servers but still illustrates
my bash illiteracy:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
#!/bin/bash

BASE=geronimo-plugin-farm-node-2.2-SNAPSHOT
<span class="code-keyword">if</span>  [ <span class="code-quote">"$1"</span>
= <span class="code-quote">"stop"</span> ] ; then
(
  cd $BASE
  <span class="code-keyword">for</span> server in server*; <span class="code-keyword">do</span>
    pid=$(ps ax | grep <span class="code-quote">"gshell-bootstrap.jar geronimo/start-server
-s $server"</span> | grep -v 'grep' | cut -c 1-6)

    <span class="code-keyword">if</span> [ $pid ] ; then   
      echo <span class="code-quote">"Killing gshell <span class="code-keyword">for</span>
$server at pid $pid"</span>
      kill -9 $pid
    fi
    pid=$(ps ax |  grep <span class="code-quote">"javaagent.*$server"</span> |
grep -v 'grep' | cut -c 1-6)

    <span class="code-keyword">if</span> [ $pid ] ; then   
      echo <span class="code-quote">"Killing $server at pid $pid"</span>
      kill -9 $pid
    fi
  done
)
fi

rm -rf $BASE
tar xzf $BASE-bin.tar.gz
(
cd $BASE

<span class="code-keyword">for</span> s in {1..5}
<span class="code-keyword">do</span> 
  sn=<span class="code-quote">"server${s}"</span>
  mkdir ${sn}
  cp -r <span class="code-keyword">var</span> ${sn}/
  <span class="code-keyword">if</span>  [ <span class="code-quote">"$1"</span>
= <span class="code-quote">"start"</span> ] ; then
    ./bin/gsh geronimo/start-server -s ${sn} -G config.substitution.PortOffset=${s}0 &gt;${sn}.log
&amp;
  fi
done
)

</pre>
</div></div></li>
	<li>Start the servers with command lines like this:
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
./bin/gsh geronimo/start-server -G server.name=server1 -G config.substitution.PortOffset=10

</pre>
</div></div>
<p>substituting appropriately for <em>server1</em> and <em>10</em></p></li>
</ol>


<h3><a name="PluginbasedFarming-Farmcontrollersetup"></a>Farm controller
setup</h3>
<ol>
	<li>Check out bits of my sandbox
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
svn co https:<span class="code-comment">//svn.apache.org/repos/asf/geronimo/sandbox/djencks/assemblies</span>
</pre>
</div></div></li>
	<li>Build the plugin and assembly
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
cd assemblies/controller-as-repo
mvn clean install
cd ../geronimo-tomcat-farm-controller
mvn clean install
cd target
tar xzf geronimo-tomcat-farm-controller-2.2-SNAPSHOT-bin.tar.gz 
./geronimo-tomcat-farm-controller-2.2-SNAPSHOT/bin/gsh geronimo/start-server
</pre>
</div></div></li>
</ol>


<h3><a name="PluginbasedFarming-Deployingasampletothefarm"></a>Deploying
a sample to the farm</h3>
<ol>
	<li>Check out and build samples (to get the plan for deployment)
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
svn co https:<span class="code-comment">//svn.apache.org/repos/asf/geronimo/samples/trunk
</span>cd trunk
mvn clean install
</pre>
</div></div></li>
	<li>In a separate terminal in geronimo-tomcat-farm-controller-2.2-SNAPSHOT start GShell(bin/gsh)
and run these commands to deploy a dependency (sample-datasource), the sample application,
and then distribute it to the farm nodes:
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
deploy/list-plugins --username system --password manager org.apache.geronimo.samples/sample-datasource/2.2-SNAPSHOT/car
# Choose the file repo based on your local maven repository
deploy/deploy /Users/david/geronimo/svn/geronimo/samples/trunk/samples/bank/bank-ear/target/bank-ear-2.2-SNAPSHOT.ear

 /Users/david/geronimo/svn/geronimo/samples/trunk/samples/bank/bank-tomcat/target/resources/META-INF/plan.xml

# Substitute actual paths to the ear and plan from your samples checkout
cluster/deploy add -f cluster1 -l pluginList1 -a org.apache.geronimo.samples/bank-tomcat/2.2-SNAPSHOT/car
 
</pre>
</div></div></li>
</ol>


<p>In the server logs for the admin server and the farm nodes you should see that the
application is installed along with all dependencies.  You should be able to see the bank
sample on the nodes at for example,
<a href="http://localhost:8120/bank/" class="external-link" rel="nofollow">http://localhost:8120/bank/</a></p>

<h3><a name="PluginbasedFarming-Removingpluginsandpluginlists"></a>Removing
plugins and plugin lists</h3>
<p>In the gsh console above try</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
cluster/deploy remove -l pluginList1 -a org.apache.geronimo.samples/bank-tomcat/2.2-SNAPSHOT/car
 
# add back the plugin...
cluster/deploy add -f cluster1 -l pluginList1 -a org.apache.geronimo.samples/bank-tomcat/2.2-SNAPSHOT/car

# remove it again
cluster/deploy remove -f cluster1 -l pluginList1   
</pre>
</div></div>

<h2><a name="PluginbasedFarming-Configurationofpluginrepositories."></a>Configuration
of plugin repositories.</h2>
<p>The example above is set up, for convenience and portability, to use the farm controller
server as the plugin repository, and to use <em>localhost</em> as the server name.
 This means no custom configuration is needed to run everything on one machine but is obviously
incomplete for any serious use.</p>

<p>The accessible plugin repositories are configured in each node in the <tt>var/config/plugin-repositories.properties</tt>
file. This needs to include the appropriate plugin repository with any needed credentials.
One way to do this is to write a non-classloader plugin to install a new <tt>var/config/plugin-repositories.properties</tt>
to replace the default.  You'd assemble a plugin node server including this additional plugin
and distribute that to the nodes.</p>

<p>In addition, the controller also needs to have this plugin repository configured
as the default in <tt>var/config/config-substitutions.properties</tt> in this
entry:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
DefaultPluginRepository=http\:<span class="code-comment">//localhost\:8080/plugin/maven-repo</span>
</pre>
</div></div> 


<h2><a name="PluginbasedFarming-Futurework"></a>Future work</h2>
<ul>
	<li>enhancing the monitoring console to find nodes using multicast discovery</li>
	<li>enhancing the car-maven-plugin to assemble servers with multiple instance support
in the assembly.</li>
</ul>



    </div>
        <div id="commentsSection" class="wiki-content pageSection">
        <div style="float: right;">
            <a href="https://cwiki.apache.org/confluence/users/viewnotifications.action"
class="grey">Change Notification Preferences</a>
        </div>
        <a href="https://cwiki.apache.org/confluence/display/GMOxDOC30/Plugin+based+Farming">View
Online</a>
        |
        <a href="https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=20645455&revisedVersion=2&originalVersion=1">View
Changes</a>
                |
        <a href="https://cwiki.apache.org/confluence/display/GMOxDOC30/Plugin+based+Farming?showComments=true&amp;showCommentArea=true#addcomment">Add
Comment</a>
            </div>
</div>
</div>
</div>
</div>
</body>
</html>

Mime
View raw message