tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Théo Chamley <theo...@mley.fr>
Subject Re: Tomcat cluster with static membership
Date Wed, 17 Dec 2014 16:22:26 GMT
On 2014-12-09 19:01, Mark Eggers wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On 12/9/2014 9:48 AM, Daniel Mikusa wrote:
>> On Tue, Dec 9, 2014 at 9:16 AM, Théo Chamley <theo.ch@mley.fr>
>> wrote:
>> 
>>> On 2014-12-08 21:22, Ameer Mawia wrote:
>>> 
>>>> Hi Theo,
>>>> 
>>>> Since you are using static membership and NOT
>>>> dynamic(multi-cast) which auto-detect members, my current
>>>> understanding says that you will have to add entry of all the
>>>> members of your cluster in each of nodes server.xml. Example:
>>>> 
>>>> My cluster has two nodes. So to configure these I had add these
>>>> two members entries in both node's server.xml(since running on
>>>> the same machine, they have varying port with same ip):
>>>> 
>>>> <Interceptor
>>>> className="org.apache.catalina.tribes.group.interceptors.
>>>> StaticMembershipInterceptor"> <Member
>>>> className="org.apache.catalina.tribes.membership.StaticMember"
>>>> port="4000" host="192.168.1.6"
>>>> uniqueId="{10,1,0,100,0,0,0,0,0,0,0,0,0,0,0,0}"/>
>>>> 
>>>> <Member
>>>> className="org.apache.catalina.tribes.membership.StaticMember"
>>>> port="4001" host="192.168.1.6"
>>>> uniqueId="{10,1,0,101,0,0,0,0,0,0,0,0,0,0,0,0}"/>
>>>> 
>>>> Regards, Ameer Mawia
>>>> 
>>>> 
>>>> On Mon, Dec 8, 2014 at 8:26 PM, Théo Chamley <theo.ch@mley.fr>
>>>> wrote:
>>>> 
>>>> Hello,
>>>>> 
>>>>> I am trying to setup a simple Tomcat cluster with static
>>>>> membership. I can't use multicast because I am on a
>>>>> virtualization environment that does not allow it.
>>>>> 
>>>>> Debian 7 Tomcat 8.0.14 Oracle JVM 1.8.0_25
>>>>> 
>>>>> Both Tomcat are ok on their own, but I can't seem to make the
>>>>> clustering work: the sessions are not replicated from one to
>>>>> another. Following the official documentation, I wrote this
>>>>> configuration :
>>>>> 
>>>>> <Engine name="Catalina" defaultHost="localhost"> <Cluster
>>>>> className="org.apache.catalina.ha.tcp. SimpleTcpCluster">
>>>>> <Manager className="org.apache.catalina.ha.session.
>>>>> BackupManager" expireSessionsOnShutdown="false"
>>>>> notifyListenersOnReplication="true" mapSendOptions="6"/>
>>>>> <Channel className="org.apache.catalina.tribes.group.
>>>>> GroupChannel"> <Receiver className="org.apache.
>>>>> catalina.tribes.transport.nio.NioReceiver" address="0.0.0.0"
>>>>> port="4110" selectorTimeout="100" maxThreads="6"/>
>>>>> 
>>>>> <Interceptor className="org.apache.catalina.tribes.group.
>>>>> interceptors.StaticMembershipInterceptor"> <Member
>>>>> className="org.apache.
>>>>> catalina.tribes.membership.StaticMember" port="4110"
>>>>> host="my.server.1" domain="staging-cluster"
>>>>> uniqueId="{1,2,3,4,5,6,7,8,9, 10,11,12,13,14,15,0}"/>
>>>>> </Interceptor>
>>>>> 
>>>>> <Sender className="org.apache.catalina.tribes.transport.
>>>>> ReplicationTransmitter"> <Transport className="org.apache.
>>>>> catalina.tribes.transport.nio.PooledParallelSender"/>
>>>>> </Sender> <Interceptor
>>>>> className="org.apache.catalina.tribes.group.
>>>>> interceptors.TcpFailureDetector"/> <Interceptor
>>>>> className="org.apache.catalina.tribes.group.
>>>>> interceptors.MessageDispatch15Interceptor"/> <Interceptor
>>>>> className="org.apache.catalina.tribes.group.
>>>>> interceptors.ThroughputInterceptor"/> </Channel> <Valve
>>>>> className="org.apache.catalina.ha.tcp. ReplicationValve"
>>>>> filter=".*\.gif|.*\.js|.*\. jpeg|.*\.jpg|.*\.png|.*\.css|.
>>>>> *\.txt"/> <ClusterListener
>>>>> className="org.apache.catalina.ha.session.
>>>>> ClusterSessionListener"/> </Cluster> [...] </Engine>
>>>>> 
>>>>> Note: I changed the host and uniqId on the
>>>>> StaticMembershipInterceptor on the other Tomcat.
>>>>> 
>>>>> This is not a network problem as I can telnet into the 4110
>>>>> port from one server to another. Also, by running a tcpdump,
>>>>> I can't see any trafic between the two servers.
>>>>> 
>>>>> The Tomcats seem to be doing something, because I have the
>>>>> following in my catalina.out:
>>>>> 
>>>>> ****** 08-Dec-2014 15:38:15.309 INFO [main]
>>>>> org.apache.catalina.ha.tcp. SimpleTcpCluster.startInternal
>>>>> Cluster is about to start 08-Dec-2014 15:38:15.312 INFO
>>>>> [main] org.apache.catalina.tribes.
>>>>> transport.ReceiverBase.bind Receiver Server Socket bound
>>>>> to:/0.0.0.0:4110 08-Dec-2014 15:38:15.328 INFO [Thread-5]
>>>>> org.apache.catalina.ha.tcp. SimpleTcpCluster.memberAdded
>>>>> Replication member
>>>>> added:org.apache.catalina.tribes.membership. StaticMember[t
>>>>> cp://my.server.1:4110,my.server.1,4110, alive=0,
>>>>> securePort=-1, UDP Port=-1, id={1 2 3 4 5 6 7 8 9 10 11 12 13
>>>>> 14 15 0 }, payload={}, command={}, domain={115 116 97 103 105
>>>>> 110 103 45 99 ...(15)}, ] 08-Dec-2014 15:38:15.330 INFO
>>>>> [main] org.apache.catalina.tribes.
>>>>> membership.McastServiceImpl.setupSocket Setting cluster mcast
>>>>> soTimeout to 500 08-Dec-2014 15:38:15.332 INFO [main]
>>>>> org.apache.catalina.tribes.
>>>>> membership.McastServiceImpl.waitForMembers Sleeping for 1000
>>>>> milliseconds to establish cluster membership, sta rt level:4
>>>>> 08-Dec-2014 15:38:16.155 INFO [Membership-MemberAdded.]
>>>>> org.apache.catalina.ha.tcp.SimpleTcpCluster.memberAdded
>>>>> Replication member
>>>>> added:org.apache.catalina.tribes.membership
>>>>> .MemberImpl[tcp://{0, 0, 0, 0}:4110,{0, 0, 0, 0},4110,
>>>>> alive=1277686, securePort=-1, UDP Port=-1, id={-22 -45 110
>>>>> -29 21 -22 75 95 -103 86 95 -119 15 48 -17 -27 }, payload={}
>>>>> , command={}, domain={}, ] 08-Dec-2014 15:38:16.259 INFO
>>>>> [Tribes-Task-Receiver-1]
>>>>> org.apache.catalina.tribes.io.BufferPool.getBufferPool
>>>>> Created a buffer pool with max size:104857600 bytes of type:
>>>>> org.apache.catalina.tribes.io. BufferPool15Impl 08-Dec-2014
>>>>> 15:38:16.332 INFO [main] org.apache.catalina.tribes.
>>>>> membership.McastServiceImpl.waitForMembers Done sleeping,
>>>>> membership established, start level:4 08-Dec-2014
>>>>> 15:38:16.335 INFO [main] org.apache.catalina.tribes.
>>>>> membership.McastServiceImpl.waitForMembers Sleeping for 1000
>>>>> milliseconds to establish cluster membership, start level:8
>>>>> 08-Dec-2014 15:38:17.335 INFO [main]
>>>>> org.apache.catalina.tribes.
>>>>> membership.McastServiceImpl.waitForMembers Done sleeping,
>>>>> membership established, start level:8 ******
>>>>> 
>>>>> Could someone, please, help me finding what I am doing
>>>>> wrong?
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> Théo C.
>>>>> 
>>>>> 
>>>>> ---------------------------------------------------------------------
>>>>> 
>>>>> 
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>> For additional commands, e-mail:
>>>>> users-help@tomcat.apache.org
>>>>> 
>>>>> 
>>>>> 
>>> Hello,
>>> 
>>> Thank you for your answer. Indeed, I need both nodes in the
>>> Interceptor, but something else is wrong. I missed it the first
>>> time, but the official documentation mentions that the
>>> TcpFailureDetector must be above the
>>> StaticMembershipInterceptor. I suspect that it is also true of
>>> the other Interceptors, but I'm not sure.
>>> 
>>> Here is my conf right now. I made some progress: I now have
>>> trafic between the two servers on the port 4110. Its regularity
>>> makes my think it's a heartbeat. However, the two Tomcat still do
>>> not share any sessions.
>>> 
>>> <Engine name="Catalina" defaultHost="localhost"> <Cluster
>>> className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
>>> channelStartOptions="3">
>>> 
>>> <Manager className="org.apache.catalina.ha.session.
>>> DeltaManager" expireSessionsOnShutdown="false"
>>> notifyListenersOnReplication="true"/>
>>> 
>>> <Channel className="org.apache.catalina.tribes.group.
>>> GroupChannel"> <Receiver className="org.apache.
>>> catalina.tribes.transport.nio.NioReceiver" address="0.0.0.0"
>>> port="4110" selectorTimeout="100" maxThreads="6"/>
>>> 
>>> <Sender className="org.apache.catalina.tribes.transport.
>>> ReplicationTransmitter"> <Transport className="org.apache.
>>> catalina.tribes.transport.nio.PooledParallelSender"/> </Sender>
>>> <Interceptor className="org.apache.catalina.tribes.group.
>>> interceptors.TcpPingInterceptor"/> <Interceptor
>>> className="org.apache.catalina.tribes.group.
>>> interceptors.TcpFailureDetector"/> <Interceptor
>>> className="org.apache.catalina.tribes.group.
>>> interceptors.MessageDispatch15Interceptor"/>
>>> 
>>> <Interceptor className="org.apache.catalina.tribes.group.
>>> interceptors.StaticMembershipInterceptor"> <Member
>>> className="org.apache. catalina.tribes.membership.StaticMember"
>>> port="4110" host="my.server.1" domain="staging-cluster"
>>> uniqueId="{1,2,3,4,5,6,7,8,9, 10,11,12,13,14,15,0}"/> <Member
>>> className="org.apache. catalina.tribes.membership.StaticMember"
>>> port="4110" host="my.server.2" domain="staging-cluster"
>>> uniqueId="{0,1,2,3,4,5,6,7,8, 9,10,11,12,13,14,15}"/>
>>> </Interceptor> </Channel> <Valve
>>> className="org.apache.catalina.ha.tcp.ReplicationValve"
>>> filter=".*\.gif|.*\.js|.*\.jpeg|.*\.jpg|.*\.png|.*\.css|.
>>> *\.txt"/> <Valve className="org.apache.catalina.ha.session.
>>> JvmRouteBinderValve"/> <ClusterListener
>>> className="org.apache.catalina.ha.session.
>>> ClusterSessionListener"/> </Cluster> [...] </Engine>
>>> 
>>> 
>>> Is there some way for me to debug what is happening?
>> 
>> 
>> Have you turned on debug level logging?  Maybe try packages like
>> "org.apache.catalina.ha" and "org.apache.catalina.tribes".  From
>> what I remember, it spits out a lot of information.
>> 
>> Dan
> 
> - From my Tomcat 7 cluster experiment (check the documentation to see 
> if
> things have changed in Tomcat 8):
> 
> # logging.properties
> 
> # all one line
> # note 5cluster.org.apache.juli.FileHandler at the end
> #
> handlers = 1catalina.org.apache.juli.FileHandler,
> 2localhost.org.apache.juli.FileHandler,
> 3manager.org.apache.juli.FileHandler,
> 4host-manager.org.apache.juli.FileHandler,
> java.util.logging.ConsoleHandler,5cluster.org.apache.juli.FileHandler
> 
> # defining the general log information
> #
> 5cluster.org.apache.juli.FileHandler.level = FINER
> 5cluster.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
> 5cluster.org.apache.juli.FileHandler.prefix = cluster.
> 
> # cluster logs - have removed farm deployer log configuration
> # beware the line wrap
> #
> org.apache.catalina.tribes.MESSAGES.level = FINE
> org.apache.catalina.tribes.MESSAGES.handlers =
> 5cluster.org.apache.juli.FileHandler
> 
> org.apache.catalina.tribes.level = FINE
> org.apache.catalina.tribes.handlers = 
> 5cluster.org.apache.juli.FileHandler
> 
> org.apache.catalina.ha.level = FINE
> org.apache.catalina.ha.handlers = 5cluster.org.apache.juli.FileHander
> 
> . . . just my two cents
> /mde/
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2
> 
> iQEcBAEBAgAGBQJUhzkVAAoJEEFGbsYNeTwtaeUIAJvtpaO//El/lz108vpgGkDZ
> g8DCIeNyhbekeo/2YI5lhTXTWsybHfwNOQEKgKhAGm76NyklJQT6GwMU6m2EmWJQ
> U+lfTGce7lX0sBMTKRBL/0VX9DV86f+p+1x0mH05gJRXQy/PfyCp/gwlE6OwLGoH
> 1Ut+qp6NJ7Bte3Z4SOt7SvqZrTCz4dGRH9+tKA5IwHfynOrJDS62p4L2+DNa2xwG
> x03V5vdCpaMq1ZpYps+wdBFQgWOfqJwOR7MO3mnP/l9ZSjWy3LGoIXbeY2OEZbKl
> 0CSBzJZYDmnzywna4YUcy5dnxl7fZJR7yDKKsWoyNiPfGOnkVaMcmQl0FlafBeQ=
> =rh+S
> -----END PGP SIGNATURE-----
> 
> ---
> This email is free from viruses and malware because avast! Antivirus
> protection is active.
> http://www.avast.com
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org

Hello,

Thank you all for your help.
Mark, things haven't changed on Tomcat8 for the logging properties, your 
configuration did work.

In the end, I had two problems:
* My client did not have a <distributable/> element in his web.xml
* More importantly, the NioReceiver was not well configured.

The "address" parameter as defined in the documentation 
(http://tomcat.apache.org/tomcat-8.0-doc/config/cluster-receiver.html)
was more that what I thought. It is not only the binding address for the 
receiver, it's also the address
that will be advertised to the other nodes of the cluster. I configured 
it to "0.0.0.0", therefore
the other node tried to connect to "0.0.0.0:<port>". You can see the 
problem...
Replacing this address by a routable one solved my problem.
Even if it is quite logical, I think the documentation could be clearer 
on this point.

Thank you again,

Théo


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message