tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Avenante <d.avena...@gmail.com>
Subject Tomcat 5.5.15 Clustering ?
Date Wed, 15 Feb 2006 01:26:49 GMT
Hi,

I' ve try to use Tomcat 5.5.15 in cluster mod. And after some work on
configuration and test
i seems to be on limit of my all possibilities ;)

I try to use two tomcat in cluster with apache web server and mod_jk as
connector.

my Apache configuration look like :


# workers.properties

ps=/

# List the workers name
worker.list=loadbalancer

# ----------------
# First worker
# ----------------
worker.node01.port=8009
worker.node01.host=agnes
worker.node01.type=ajp13
worker.node01.lbfactor=1
#worker.node01.domain=cluster1
# ----------------
# Second worker
# ----------------
worker.node02.port=9009
worker.node02.host=ovea-inspiron
worker.node02.type=ajp13
worker.node02.lbfactor=1
#worker.node02.domain=cluster1
# ----------------------
# Load Balancer worker
# ----------------------
worker.loadbalancer.type=lb
worker.loadbalancer.sticky_session=false
worker.loadbalancer.balanced_workers=node01,node02

As you can see i use two server ('agnes' IP : 192.168.2.102 and
'ovea-inspiron' IP : 192.168.2.103)
I' ve coded a little aplication who read un file and create a sessions if
session not exist.

If i use the system in mod load balancing (with no cluster mod activate)
all is great (my config is of course worker.loadbalancer.sticky_session=true
)

So system run with session affinitu but the two server are accessible ;)

Now i try to configure as cluster mode with session replication
(my config is now worker.loadbalancer.sticky_session=3Dfalse)

I configure my to server.xml like that :
agnes (IP : 192.168.2.102)

<Server port="8005" shutdown="SHUTDOWN">
  <GlobalNamingResources>
    <Resource name="UserDatabase" auth="Container"
                   type="org.apache.catalina.UserDatabase"
                   description="User database that can be updated and saved"
                   factory="
org.apache.catalina.users.MemoryUserDatabaseFactory"
                   pathname="conf/tomcat-users.xml" />

  </GlobalNamingResources>

  <Service name="Catalina">
    <Connector port="8080" maxHttpHeaderSize="8192"
                      maxThreads="150" minSpareThreads="25"
                      maxSpareThreads="75"
                      enableLookups="false" redirectPort="8443"
acceptCount="100"
                      connectionTimeout="20000" disableUploadTimeout="true"
/>

    <Connector port="8009" enableLookups="false"
                      redirectPort="8443" protocol="AJP/1.3" />

    <Engine name="Catalina" defaultHost="localhost" jvmRoute="node01" >

      <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
                   resourceName="UserDatabase" />

      <Host name="localhost" appBase="webapps"
               unpackWARs="true" autoDeploy="true"
               xmlValidation="false" xmlNamespaceAware="false">
     <Cluster className="org.apache.catalina.cluster.tcp.SimpleTcpCluster"
                  doClusterLog="true"
                  clusterLogName="clusterlog"
                  manager.className="
org.apache.catalina.cluster.session.DeltaManager"
                  manager.expireSessionsOnShutdown="false"
                  manager.notifyListenersOnReplication="true"
                  manager.notifySessionListenersOnReplication="true"
                  manager.sendAllSessions="true"
                  manager.sendAllSessionsSize="500"
                  manager.sendAllSessionsWaitTime="20">

        <Membership className="
org.apache.catalina.cluster.mcast.McastService"
                             mcastBindAddress="192.168.2.102"
                             mcastAddr="224.0.0.1"
                             mcastPort="45564"
                             mcastFrequency="1000"
                             mcastDropTime="30000"/>

        <Receiver className="
org.apache.catalina.cluster.tcp.ReplicationListener"
                         tcpListenAddress="192.168.2.102"
                         tcpListenPort="4000"
                         tcpSelectorTimeout="100"
                         tcpThreadCount="6" />

        <Sender className="
org.apache.catalina.cluster.tcp.ReplicationTransmitter"
                      replicationMode="fastasyncqueue"
                      doTransmitterProcessingStats="true"
                      doProcessingStats="true"
                      doWaitAckStats="true"
                      queueTimeWait="true"
                      queueDoStats="true"
                      queueCheckLock="true"
                      ackTimeout="15000"
                      waitForAck="true"
                      keepAliveTimeout="80000"
                      keepAliveMaxRequestCount="-1" />

      </Host>
    </Engine>
  </Service>
</Server>


ovea-inspiron (IP : 192.168.2.103)
<Server port="9005" shutdown="SHUTDOWN">

  <GlobalNamingResources>
    <Resource name="UserDatabase" auth="Container"
                     type="org.apache.catalina.UserDatabase"
                     description="User database that can be updated and
saved"
                     factory="
org.apache.catalina.users.MemoryUserDatabaseFactory"
                     pathname="conf/tomcat-users.xml" />
  </GlobalNamingResources>

  <Service name="Catalina">

   <Connector port="9080" maxHttpHeaderSize="8192"
                     maxThreads="150" minSpareThreads="25"
                     maxSpareThreads="75"
                     enableLookups="false" redirectPort="8443"
                     acceptCount="100"
                     connectionTimeout="20000" disableUploadTimeout="true"
/>

   <Connector port="9009" enableLookups="false"
                     redirectPort="8443" protocol="AJP/1.3" />

   <Engine name="Catalina" defaultHost="localhost" jvmRoute="node02" >
     <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
                  resourceName="UserDatabase"/>

     <Host name="localhost" appBase="webapps"
              unpackWARs="true" autoDeploy="true"
              xmlValidation="false" xmlNamespaceAware="false">

     <Cluster className="org.apache.catalina.cluster.tcp.SimpleTcpCluster"
                  doClusterLog="true"
                  clusterLogName="clusterlog"
                  manager.className="
org.apache.catalina.cluster.session.DeltaManager"
                  manager.expireSessionsOnShutdown="false"
                  manager.notifyListenersOnReplication="true"
                  manager.notifySessionListenersOnReplication="true"
                  manager.sendAllSessions="true"
                  manager.sendAllSessionsSize="500"
                  manager.sendAllSessionsWaitTime="20">

        <Membership className="
org.apache.catalina.cluster.mcast.McastService"
                             mcastBindAddress="192.168.2.103"
                             mcastAddr="224.0.0.1"
                             mcastPort="45564"
                             mcastFrequency="1000"
                             mcastDropTime="30000"/>

        <Receiver className="
org.apache.catalina.cluster.tcp.ReplicationListener"
                        tcpListenAddress="192.168.2.103"
                        tcpListenPort="4000"
                        tcpSelectorTimeout="100"
                        tcpThreadCount="6" />

        <Sender className="
org.apache.catalina.cluster.tcp.ReplicationTransmitter"
                      replicationMode="fastasyncqueue"
                      doTransmitterProcessingStats="true"
                      doProcessingStats="true"
                      doWaitAckStats="true"
                      queueTimeWait="true"
                      queueDoStats="true"
                      queueCheckLock="true"
                      ackTimeout="15000"
                      waitForAck="true"
                      keepAliveTimeout="80000"
                      keepAliveMaxRequestCount="-1" />

      </Host>
    </Engine>
  </Service>
</Server>


I'm user linux so i've compiled my kernel with multicat kernel option
I add the rout like rooute add -host 224.0.0.1 dev eth0

N.B : why 224.0.0.1 => http://www.iana.org/assignments/multicast-addresses

So route is :
ALL-SYSTEMS.MCA *     255.255.255.255 UH    0      0        0 eth0
192.168.2.0     *              255.255.255.0     U      0      0        0
eth0
loopback        *               255.0.0.0            U      0      0
 0 lo
default         .                  0.0.0.0               UG    0      0
   0 eth0

now i start the two tomcat with the same application in webapp directories
with tcpdump i verify the multicasting btewen my server .. so :

19:39:17.809978 IP 192.168.2.102.45564 > all-systems.mcast.net.45564:UDP,
length 56
19:39:18.546034 IP ovea-inspiron.45564 > all-systems.mcast.net.45564:UDP,
length 56
19:39:18.814418 IP 192.168.2.102.45564 > all-systems.mcast.net.45564UDP,
length 56
19:39:19.790470 IP ovea-inspiron.45564 > all-systems.mcast.net.45564:UDP,
length 56
19:39:19.818691 IP 192.168.2.102.45564 > all-systems.mcast.net.45564:UDP,
length 56
19:39:20.826650 IP 192.168.2.102.45564 > all-systems.mcast.net.45564:UDP,
length 56
19:39:20.943933 IP ovea-inspiron.45564 > all-systems.mcast.net.45564:UDP,
length 56
19:39:21.806813 arp who-has . tell 192.168.2.102
19:39:21.806972 arp reply . is-at 00:04:e2:b6:65:0a (oui Unknown)
19:39:21.831048 IP 192.168.2.102.45564 > all-systems.mcast.net.45564:UDP,
length 56
19:39:22.164812 IP ovea-inspiron.45564 > all-systems.mcast.net.45564:UDP,
length 56
19:39:22.835240 IP 192.168.2.102.45564 > all-systems.mcast.net.45564:UDP,
length 56

So all semms to be OK but i've

Feb 14, 2006 6:33:26 PM org.apache.catalina.cluster.tcp.SimpleTcpCluster st
art
INFO: Cluster is about to start
Feb 14, 2006 6:33:26 PM
org.apache.catalina.cluster.tcp.SimpleTcpClustercreateDefaultClusterValves
INFO: Add Default ClusterValves at cluster localhost
Feb 14, 2006 6:33:26 PM
org.apache.catalina.cluster.tcp.SimpleTcpClustercreateDefaultClusterListener
INFO: Add Default ClusterListener at cluster localhost
Feb 14, 2006 6:33:26 PM
org.apache.catalina.cluster.tcp.ReplicationTransmitter start
INFO: Start ClusterSender at cluster Catalina:type=3DCluster,host=3Dlocalhost
with name Catalina:type=3DClusterSender,host=3Dlocalhost
Feb 14, 2006 6:33:26 PM
org.apache.catalina.cluster.mcast.McastServiceImplsetupSocket
INFO: Setting multihome multicast interface to:/192.168.2.102
Feb 14, 2006 6:33:26 PM org.apache.catalina.cluster.mcast.McastService start
INFO: Sleeping for 4000 secs to establish cluster membership
Feb 14, 2006 6:33:30 PM
org.apache.catalina.cluster.mcast.McastServiceregisterMBean
INFO: membership mbean registered
(Catalina:type=3DClusterMembership,host=3Dlocalhost)
Feb 14, 2006 6:33:31 PM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive cluster-1.0-SNAPSHOT.war
Feb 14, 2006 6:33:31 PM org.apache.catalina.cluster.session.DeltaManager st
art
INFO: Register manager /cluster-1.0-SNAPSHOT to cluster element Host with
name localhost
Feb 14, 2006 6:33:31 PM org.apache.catalina.cluster.session.DeltaManager st
art
INFO: Starting clustering manager at /cluster-1.0-SNAPSHOT
Feb 14, 2006 6:33:31 PM
org.apache.catalina.cluster.session.DeltaManagergetAllClusterSessions
INFO: Manager [/cluster-1.0-SNAPSHOT]: skipping state transfer. No members
active in cluster group.

As you can see in the last line cluster member are not found !!!!

What's the hell ;)

When i test my application i've always the sequences :

   This page is rendered on server : *Cluster Node - 01 (One) New session is
created with saved value ....UserId : 123456
   This page is rendered on server : *Cluster Node - 02 (Two) New session is
created with saved value ....UserId : 123456
   This page is rendered on server : *Cluster Node - 01 (One) New session is
created with saved value ....UserId : 123456

So the request is well balanced but a new session is alway created !!!!!

Some ideas ...

Thank you guys

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message