qpid-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From conflue...@apache.org
Subject [CONF] Apache Qpid > Starting a cluster
Date Mon, 24 Aug 2009 17:49:00 GMT
<html>
<head>
    <base href="http://cwiki.apache.org/confluence">
            <link rel="stylesheet" href="/confluence/s/1519/1/1/_/styles/combined.css?spaceKey=qpid&amp;forWysiwyg=true"
type="text/css">
    </head>
<body style="background-color: white" bgcolor="white">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
     <h2><a href="http://cwiki.apache.org/confluence/display/qpid/Starting+a+cluster">Starting
a cluster</a></h2>
     <h4>Page <b>edited</b> by             <a href="http://cwiki.apache.org/confluence/display/~aconway">Alan
Conway</a>
    </h4>
     
          <br/>
     <div class="notificationGreySide">
         <h1><a name="Startingacluster-RunningaQpiddcluster"></a>Running
a Qpidd cluster</h1>

<p>There are several pre-requisites to running a qpidd cluster:</p>

<h2><a name="Startingacluster-Installandconfigureopenais%2Fcorosync"></a>Install
and configure openais/corosync</h2>

<p>Qpid clustering uses a multicast protocol provided by the corosync (formerly called
openais) library. Install whichever is available on your OS.<br/>
E.g. in fedora10: yum install corosync.</p>

<p>The configuration file is /etc/ais/openais.conf on openais, /etc/corosync.conf on
early corosync versions and /etc/corosync/corosync.conf on recent corosync versions. You will
need to edit the default file created when you installed</p>

<p>Here is an example, with places marked that you will<br/>
change.  ( Below, I will describe how to change the file. )</p>

<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
# Please read the openais.conf.5 manual page

totem {
        version: 2
        secauth: off
        threads: 0
        <span class="code-keyword">interface</span> {
                ringnumber: 0
                ## You must change <span class="code-keyword">this</span> address
##
                bindnetaddr: 20.0.100.0
                mcastaddr: 226.94.32.36
                mcastport: 5405
        }
}

logging {
        debug: off
	timestamp: on
	to_file: yes
	logfile: /tmp/aisexec.log
}

amf {
     	mode: disabled
}
</pre>
</div></div>

<p>You must sent the bindnetaddr entry in the configuration file to the network address
of your network interface. This must be a real network interface, not the loopback address
127.0.0.1</p>

<p>You can find your network interface by running ifconfig. This will list the address
and the mask, e.g.</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
inet addr:20.0.20.32  Bcast:20.0.20.255  Mask:255.255.255.0
</pre>
</div></div>

<p>The bindnetaddr is the logical AND of the inet addr and mask values, in the example
above 20.0.20.0</p>


<h2><a name="Startingacluster-Openyourfirewall"></a>Open your firewall</h2>

<p>In the above example file, I use mcastport 5405.<br/>
This implies that your firewall must allow UDP<br/>
protocol over port 5405, or that you disable the firewall</p>

<h2><a name="Startingacluster-Usetheproperidentity."></a>Use the proper
identity.</h2>

<p>The qpidd process must be started with the correct identity in order to use the corosync/openais
library.</p>

<p>For openais and early corosync versions the installation of openAIS/corosync on your
system will create a new<br/>
group called "ais".  The user that starts the qpidd processes of the cluster<br/>
must have "ais" as its effective group id. You can create a user specifically for this purpose
with ais as the primary group, or <br/>
a user that has ais as a secondary group can use "newgrp" to set the primary group to ais
when running qpidd.</p>

<p>For recent corosync versions you no longer need to set your group to "ais" but you
do need to create a file in /etc/corosync/uidgid.d/ to allow access for whatever user/group
ID you want to use. For example create /etc/corosync/uidgid.d/qpid  th the contents:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
uidgid {
   uid: qpid
   gid: qpid
}
</pre>
</div></div>


<h2><a name="Startingacluster-StartingaCluster"></a>Starting a Cluster</h2>

<p>To be a member of a cluster you must pass the &#45;-cluster-name argument to
qpidd. This is the only required option to join a&nbsp; cluster, other options can be
set as for a normal qpidd.</p>


<p>For example to start a cluster of 3 brokers on the current host<br/>
Here is an example of starting a cluster of 3 members, all on the current host but with different
ports and different log files:</p>

<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
qpidd -p5672 --cluster-name=MY_CLUSTER --log-output=cluster0.log -d --no-data-dir --auth=no
qpidd -p5673 --cluster-name=MY_CLUSTER --log-output=cluster0.log -d --no-data-dir --auth=no
qpidd -p5674 --cluster-name=MY_CLUSTER --log-output=cluster0.log -d --no-data-dir --auth=no
</pre>
</div></div>
<p>In a deployed system, cluster members will normally be on different hosts but for
development its useful to be able to create a cluster on a single host.</p>

<h2><a name="Startingacluster-SELinuxconflicts"></a>SELinux conflicts</h2>

<p>Developers will often start openais/corosync as a service like this:</p>

<p>service openais start</p>

<p>But will then will start a cluster-broker without using the service script like this:</p>

<p>/usr/sbin/qpidd &#45;-cluster-name my_cluster ...</p>

<p>If SELinux is in enforcing mode this may cause qpidd to hang due because of the different
SELinux contexts.<br/>
There are 3 ways to resolve this:</p>
<ul class="alternate" type="square">
	<li>run both qpidd and openais/corosync as services.</li>
	<li>run both qpidd and openais/corosync as user processes.</li>
	<li>make selinux permissive:</li>
</ul>


<p>To check what mode selinux is running:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
# getenforce
</pre>
</div></div>

<p>To change the mode:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
# setenforce permissive
</pre>
</div></div>

<p>Note that in a deployed system both openais/corosync and qpidd should be started
as services, in which case there is no problem with SELinux running in enforcing mode.</p>

<h2><a name="Startingacluster-Troubleshootingchecklist."></a>Troubleshooting
checklist.</h2>

<p>If you have trouble starting your cluster, make sure that:</p>

<p>1. You have edited the correct openais/corosync configuration file and set bindnetaddr
correctly<br/>
1. Your firewall allows UDP on the openais/corosync mcastport<br/>
2. Your effective group is "ais" (openais/old corosync) or you have created an appropriate
ID file (new corosync)<br/>
3. Your firewall allows TCP on the ports used by qpidd.<br/>
4. If you're starting openais as a service but running qpidd directly, ensure selinux is in
permissive mode</p>

     </div>
     <div id="commentsSection" class="wiki-content pageSection">
       <div style="float: right;">
            <a href="http://cwiki.apache.org/confluence/users/viewnotifications.action"
class="grey">Change Notification Preferences</a>
       </div>

       <a href="http://cwiki.apache.org/confluence/display/qpid/Starting+a+cluster">View
Online</a>
       |
       <a href="http://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=2326977&revisedVersion=3&originalVersion=2">View
Change</a>
              |
       <a href="http://cwiki.apache.org/confluence/display/qpid/Starting+a+cluster?showComments=true&amp;showCommentArea=true#addcomment">Add
Comment</a>
            </div>
</div>
</div>
</div>
</div>
</body>
</html>

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:commits-subscribe@qpid.apache.org


Mime
View raw message