qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pavel Moravec <pmora...@redhat.com>
Subject Re: Qpid clustering with cman
Date Thu, 08 Dec 2011 16:24:04 GMT
Hi,
this is a known issue, see https://issues.apache.org/jira/browse/QPID-3483.

Some background: corosync has not authenticated qpidd process to access its CPG methods. Without
CMAN in use, the access is allowed by /etc/corosync/uidgid.d/qpidd file. However, CMAN configuration
(cluster.conf) simply ignores the file at all, not providing an option to set it up within
the conf file.

Workaround: run qpidd under root (as root user has access to corosync CPG).

Kind regards,
Pavel Moravec


----- Original Message -----
> From: "David Zhou" <daydayya@gmail.com>
> To: users@qpid.apache.org
> Sent: Thursday, December 8, 2011 8:33:44 AM
> Subject: Qpid clustering with cman
> 
> (nabble link:
> http://apache-qpid-users.2158936.n2.nabble.com/Qpid-clustering-with-cman-td7073352.html
> )
> 
> Hi all,
> 
> I set up a 2-nodes Qpid cluster (0.12) which is synchronized by
> corosync
> (1.4.2).
> The cluster works fine except running with cman to avoid split-brain.
> It's said that settings of corosync.conf would be replaced by
> cluster.conf,
> but from the following trial and error it seems that corosync doesn't
> fetch
> settings from cluster.conf.
> I wonder there are problems with my configurations. Could you please
> help
> me out? Thank you!!
> 
> First here apply the configuration sample from Qpid
> wiki<https://cwiki.apache.org/qpid/configuring-qpidd-with-cluster-manager.html>
> :
> 
> --cluster.conf------------------------------------------
> <?xml version="1.0" ?>
> <cluster name="David" config_version="1">
>    <cman two_node="1" expected_votes="1">
>    </cman>
>    <clusternodes>
>      <clusternode name="c1q1" nodeid="1">
>      </clusternode>
>      <clusternode name="c1q2" nodeid="2">
>      </clusternode>
>    </clusternodes>
>    <rm>
>      <failoverdomains>
>        <failoverdomain name="only_broker1" nofailback="0" ordered="0"
> restricted="1">
>          <failoverdomainnode name="c1q1" priority="1"/>
>        </failoverdomain>
>        <failoverdomain name="only_broker2" nofailback="0" ordered="0"
> restricted="1">
>          <failoverdomainnode name="c1q2" priority="1"/>
>        </failoverdomain>
>      </failoverdomains>
>      <resources>
>        <script name="qpidd" file="/etc/init.d/qpidd" />
>      </resources>
>      <service name="qpidd_broker1" domain="only_broker1">
>        <script ref="qpidd" />
>      </service>
>      <service name="qpidd_broker2" domain="only_broker2">
>        <script ref="qpidd" />
>      </service>
>    </rm>
> </cluster>
> -------------------------------------------------------
> 
> => Starting cman OK, but starting qpidd get the following error:
> 
> --log-------------------------------------------------
> Dec  7 19:02:45 c1q2 qpidd[6893]: notice Journal "TplStore": Created
> Dec  7 19:02:45 c1q2 qpidd[6893]: notice Store module initialized;
> store-dir=/var/spool/qpid
> Dec  7 19:02:45 c1q2 qpidd[6893]: notice Initializing CPG
> Dec  7 19:02:45 c1q2 corosync[6572]:   [IPC   ] Invalid IPC
> credentials.
> Dec  7 19:02:45 c1q2 qpidd[6893]: critical Unexpected error: Failed
> to
> initialize CPG.: security (100)
> -------------------------------------------------------
> 
> I think it's because the bindnetaddr is not set in cluster.conf
> (Without
> cman, corosync + qpidd works fine), so I add the syntax found at
> here<https://bugzilla.redhat.com/show_bug.cgi?id=613880>
> 
> --cluster.conf------------------------------------------
> <?xml version="1.0" ?>
> <cluster name="David" config_version="1">
>    <cman two_node="1" expected_votes="1">
>    </cman>
>    <totem>
>      <interface ringnumber="0" bindnetaddr="192.168.0.0"
> mcastaddr="226.94.1.1" mcastport="5405" />
>    </totem>
>    <clusternodes>
>      <clusternode name="c1q1" nodeid="1">
>      </clusternode>
>      <clusternode name="c1q2" nodeid="2">
>      </clusternode>
>    </clusternodes>
>    <rm>
>      <failoverdomains>
>        <failoverdomain name="only_broker1" nofailback="0" ordered="0"
> restricted="1">
>          <failoverdomainnode name="c1q1" priority="1"/>
>        </failoverdomain>
>        <failoverdomain name="only_broker2" nofailback="0" ordered="0"
> restricted="1">
>          <failoverdomainnode name="c1q2" priority="1"/>
>        </failoverdomain>
>      </failoverdomains>
>      <resources>
>        <script name="qpidd" file="/etc/init.d/qpidd" />
>      </resources>
>      <service name="qpidd_broker1" domain="only_broker1">
>        <script ref="qpidd" />
>      </service>
>      <service name="qpidd_broker2" domain="only_broker2">
>        <script ref="qpidd" />
>      </service>
>    </rm>
> </cluster>
> -------------------------------------------------------
> 
> The result is not OK. corosync finds no multicast address specified.
> 
> --log--------------------------------------------------
> Dec  8 11:05:08 c1q2 corosync[7465]:   [MAIN  ] Corosync Cluster
> Engine ('1.4.2'): started and ready to provide service.
> Dec  8 11:05:08 c1q2 corosync[7465]:   [MAIN  ] Corosync built-in
> features: nss
> Dec  8 11:05:08 c1q2 corosync[7465]:   [MAIN  ] Successfully read
> config from /etc/cluster/cluster.conf
> Dec  8 11:05:08 c1q2 corosync[7465]:   [MAIN  ] Successfully parsed
> cman config
> Dec  8 11:05:08 c1q2 corosync[7465]:   [MAIN  ] Successfully
> configured openais services to load
> Dec  8 11:05:08 c1q2 corosync[7465]:   [MAIN  ] parse error in
> config:
> No multicast address specified
> Dec  8 11:05:08 c1q2 corosync[7465]:   [MAIN  ] Corosync Cluster
> Engine exiting with status 8 at main.c:1708.
> --------------------------------------------------------
> 
> Then I change the settings according to the *Multicast* section
> described
> here <http://sourceware.org/cluster/doc/usage.txt>
> 
> --cluster.conf------------------------------------------
> <?xml version="1.0" ?>
> <cluster name="David" config_version="1">
>    <cman two_node="1" expected_votes="1">
>      <multicast addr="226.94.1.1"/>
>    </cman>
>    <totem>
>      <interface bindnetaddr="192.168.0.0" />
>    </totem>
>    <clusternodes>
>      <clusternode name="c1q1" nodeid="1">
>        <multicast addr="226.94.1.1" interface="eth0"/>
>      </clusternode>
>      <clusternode name="c1q2" nodeid="2">
>        <multicast addr="226.94.1.1" interface="eth0"/>
>      </clusternode>
>    </clusternodes>
>    <fencedevices>
>    </fencedevices>
>    <rm>
>      <failoverdomains>
>        <failoverdomain name="only_broker1" nofailback="0" ordered="0"
> restricted="1">
>          <failoverdomainnode name="c1q1" priority="1"/>
>        </failoverdomain>
>        <failoverdomain name="only_broker2" nofailback="0" ordered="0"
> restricted="1">
>          <failoverdomainnode name="c1q2" priority="1"/>
>        </failoverdomain>
>      </failoverdomains>
>      <resources>
>        <script name="qpidd" file="/etc/init.d/qpidd" />
>      </resources>
>      <service name="qpidd_broker1" domain="only_broker1">
>        <script ref="qpidd" />
>      </service>
>      <service name="qpidd_broker2" domain="only_broker2">
>        <script ref="qpidd" />
>      </service>
>    </rm>
> </cluster>
> -------------------------------------------------------
> 
> It still not OK as the following shows:
> 
> --log--------------------------------------------------
> Dec  8 11:51:23 c1q2 corosync[7936]:   [MAIN  ] Corosync Cluster
> Engine ('1.4.2'): started and ready to provide service.
> Dec  8 11:51:23 c1q2 corosync[7936]:   [MAIN  ] Corosync built-in
> features: nss
> Dec  8 11:51:23 c1q2 corosync[7936]:   [MAIN  ] Successfully read
> config from /etc/cluster/cluster.conf
> Dec  8 11:51:23 c1q2 corosync[7936]:   [MAIN  ] Successfully parsed
> cman config
> Dec  8 11:51:23 c1q2 corosync[7936]:   [MAIN  ] Successfully
> configured openais services to load
> Dec  8 11:51:23 c1q2 corosync[7936]:   [MAIN  ] parse error in
> config:
> No multicast address specified
> Dec  8 11:51:23 c1q2 corosync[7936]:   [MAIN  ] Corosync Cluster
> Engine exiting with status 8 at main.c:1708.
> --------------------------------------------------------
> 
> --console----------------------------------------------
> root@c1q2:/etc/cluster# service cman start
> Starting cluster:
>    Checking Network Manager... [  OK  ]
>    Global setup... [  OK  ]
>    Loading kernel modules... [  OK  ]
>    Mounting configfs... [  OK  ]
>    Starting cman... tempfile:11: element multicast: Relax-NG validity
> error : Element clusternode has extra content: multicast
> tempfile:10: element clusternode: Relax-NG validity error : Element
> clusternodes has extra content: clusternode
> Configuration fails to validate
> corosync died: Error, reason code is 8
> [FAILED]
> -------------------------------------------------------
> 
> 
> Thank you for you help!!
> 
> 
> --
> View this message in context:
> http://apache-qpid-users.2158936.n2.nabble.com/Qpid-clustering-with-cman-tp7073493p7073493.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org


Mime
View raw message