qpid-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From conflue...@apache.org
Subject [CONF] Apache Qpid > Starting a cluster
Date Tue, 15 Jan 2013 16:39:00 GMT
<html>
<head>
    <base href="https://cwiki.apache.org/confluence">
            <link rel="stylesheet" href="/confluence/s/2042/9/21/_/styles/combined.css?spaceKey=qpid&amp;forWysiwyg=true"
type="text/css">
    </head>
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
    <h2><a href="https://cwiki.apache.org/confluence/display/qpid/Starting+a+cluster">Starting
a cluster</a></h2>
    <h4>Page <b>edited</b> by             <a href="https://cwiki.apache.org/confluence/display/~aconway">Alan
Conway</a>
    </h4>
        <br/>
                         <h4>Changes (1)</h4>
                                 
    
<div id="page-diffs">
                    <table class="diff" cellpadding="0" cellspacing="0">
    
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >h2. Starting a Cluster <br>
<br></td></tr>
            <tr><td class="diff-added-lines" style="background-color: #dfd;">Make
sure the openais or corosync service has been started. <br> <br></td></tr>
            <tr><td class="diff-unchanged" >To be a member of a cluster you must
pass the \--cluster-name argument to qpidd. This is the only required option to join a&amp;nbsp;
cluster, other options can be set as for a normal qpidd. <br> <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
    
            </table>
    </div>                            <h4>Full Content</h4>
                    <div class="notificationGreySide">
        <h1><a name="Startingacluster-RunningaQpiddcluster"></a>Running
a Qpidd cluster</h1>

<p>There are several pre-requisites to running a qpidd cluster:</p>

<h2><a name="Startingacluster-Installandconfigureopenais%2Fcorosync"></a>Install
and configure openais/corosync</h2>

<p>Qpid clustering uses a multicast protocol provided by the corosync (formerly called
openais) library. Install whichever is available on your OS.<br/>
E.g. in fedora10: yum install corosync.</p>

<p>The configuration file is /etc/ais/openais.conf on openais, /etc/corosync.conf on
early corosync versions and /etc/corosync/corosync.conf on recent corosync versions. You will
need to edit the default file created when you installed</p>

<p>Here is an example, with places marked that you will<br/>
change.  ( Below, I will describe how to change the file. )</p>

<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
# Please read the openais.conf.5 manual page

totem {
        version: 2
        secauth: off
        threads: 0
        <span class="code-keyword">interface</span> {
                ringnumber: 0
                ## You must change <span class="code-keyword">this</span> address
##
                bindnetaddr: 20.0.100.0
                mcastaddr: 226.94.32.36
                mcastport: 5405
        }
}

logging {
        debug: off
	timestamp: on
	to_file: yes
	logfile: /tmp/aisexec.log
}

amf {
     	mode: disabled
}
</pre>
</div></div>

<p>You must sent the bindnetaddr entry in the configuration file to the network address
of your network interface. This must be a real network interface, not the loopback address
127.0.0.1</p>

<p>You can find your network interface by running ifconfig. This will list the address
and the mask, e.g.</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
inet addr:20.0.20.32  Bcast:20.0.20.255  Mask:255.255.255.0
</pre>
</div></div>

<p>The bindnetaddr is the logical AND of the inet addr and mask values, in the example
above 20.0.20.0</p>


<h2><a name="Startingacluster-Openyourfirewall"></a>Open your firewall</h2>

<p>In the above example file, I use mcastport 5405.<br/>
This implies that your firewall must allow UDP<br/>
protocol over port 5405, or that you disable the firewall</p>

<h2><a name="Startingacluster-Usetheproperidentity."></a>Use the proper
identity.</h2>

<p>The qpidd process must be started with the correct identity in order to use the corosync/openais
library.</p>

<p>For openais and early corosync versions the installation of openAIS/corosync on your
system will create a new<br/>
group called "ais".  The user that starts the qpidd processes of the cluster<br/>
must have "ais" as its effective group id. You can create a user specifically for this purpose
with ais as the primary group, or<br/>
a user that has ais as a secondary group can use "newgrp" to set the primary group to ais
when running qpidd.</p>

<p>For recent corosync versions you no longer need to set your group to "ais" but you
do need to create a file in /etc/corosync/uidgid.d/ to allow access for whatever user/group
ID you want to use. For example create /etc/corosync/uidgid.d/qpid  th the contents:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
uidgid {
   uid: qpid
   gid: qpid
}
</pre>
</div></div>


<h2><a name="Startingacluster-StartingaCluster"></a>Starting a Cluster</h2>

<p>Make sure the openais or corosync service has been started.</p>

<p>To be a member of a cluster you must pass the &#45;-cluster-name argument to
qpidd. This is the only required option to join a&nbsp; cluster, other options can be
set as for a normal qpidd.</p>


<p>For example to start a cluster of 3 brokers on the current host<br/>
Here is an example of starting a cluster of 3 members, all on the current host but with different
ports and different log files:</p>

<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
qpidd -p5672 --cluster-name=MY_CLUSTER --log-output=cluster0.log -d --no-data-dir --auth=no
qpidd -p5673 --cluster-name=MY_CLUSTER --log-output=cluster0.log -d --no-data-dir --auth=no
qpidd -p5674 --cluster-name=MY_CLUSTER --log-output=cluster0.log -d --no-data-dir --auth=no
</pre>
</div></div>
<p>In a deployed system, cluster members will normally be on different hosts but for
development its useful to be able to create a cluster on a single host.</p>

<h2><a name="Startingacluster-SELinuxconflicts"></a>SELinux conflicts</h2>

<p>Developers will often start openais/corosync as a service like this:</p>

<p>service openais start</p>

<p>But will then will start a cluster-broker without using the service script like this:</p>

<p>/usr/sbin/qpidd &#45;-cluster-name my_cluster ...</p>

<p>If SELinux is in enforcing mode this may cause qpidd to hang due because of the different
SELinux contexts.<br/>
There are 3 ways to resolve this:</p>
<ul class="alternate" type="square">
	<li>run both qpidd and openais/corosync as services.</li>
	<li>run both qpidd and openais/corosync as user processes.</li>
	<li>make selinux permissive:</li>
</ul>


<p>To check what mode selinux is running:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
# getenforce
</pre>
</div></div>

<p>To change the mode:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
# setenforce permissive
</pre>
</div></div>

<p>Note that in a deployed system both openais/corosync and qpidd should be started
as services, in which case there is no problem with SELinux running in enforcing mode.</p>

<h2><a name="Startingacluster-Troubleshootingchecklist."></a>Troubleshooting
checklist.</h2>

<p>If you have trouble starting your cluster, make sure that:</p>

<ol>
	<li>You have edited the correct openais/corosync configuration file and set bindnetaddr
correctly</li>
	<li>Your firewall allows UDP on the openais/corosync mcastport</li>
	<li>Your effective group is "ais" (openais/old corosync) or you have created an appropriate
ID file (new corosync)</li>
	<li>Your firewall allows TCP on the ports used by qpidd.</li>
	<li>If you're starting openais as a service but running qpidd directly, ensure selinux
is in permissive mode</li>
</ol>




<h2><a name="Startingacluster-TroubleshootingFAQ."></a>Troubleshooting FAQ.</h2>

<ul>
	<li>If I got  "Daemon startup failed: Cluster-ID mismatch. stores belong to different
clusters.", even if I did check name-cluster
	<ol>
		<li>Clean all filesystem database, with <b>&#45;-truncate yes</b></li>
		<li>Delete all file with <b>rm &#45;rf /var/lib/qpidd/&#42;</b>
(depending data-dir option)</li>
	</ol>
	</li>
</ul>


<ul>
	<li>If I got "Starting Qpid AMQP daemon: Daemon startup failed: Queue XXXXX: recoverQueues()
failed: jexception 0x0c00 jinf::validate() threw JERR_JINF_CVALIDFAIL: Journal compatibility
validation failure. (File "/var/lib/qpidd/rhm/jrnl/NNNN/XXXXX/JournalData.jinf": RHM_JDAT_VERSION
mismatch:  (MessageStoreImpl.cpp:820)"
	<ol>
		<li>Your datastore may be corrupt after a insuffisant disk space error</li>
		<li>Resize your filesystem up</li>
		<li>Clean all filesystem database, with <b>&#45;-truncate yes</b></li>
		<li>Delete all file with <b>rm &#45;rf /var/lib/qpidd/&#42;</b>
(depending data-dir option)</li>
	</ol>
	</li>
</ul>

    </div>
        <div id="commentsSection" class="wiki-content pageSection">
        <div style="float: right;">
            <a href="https://cwiki.apache.org/confluence/users/viewnotifications.action"
class="grey">Change Notification Preferences</a>
        </div>
        <a href="https://cwiki.apache.org/confluence/display/qpid/Starting+a+cluster">View
Online</a>
        |
        <a href="https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=2326977&revisedVersion=6&originalVersion=5">View
Changes</a>
                |
        <a href="https://cwiki.apache.org/confluence/display/qpid/Starting+a+cluster?showComments=true&amp;showCommentArea=true#addcomment">Add
Comment</a>
            </div>
</div>
</div>
</div>
</div>
</body>
</html>

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@qpid.apache.org
For additional commands, e-mail: commits-help@qpid.apache.org


Mime
View raw message