tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Rellis <john.d.rel...@gmail.com>
Subject Static Membership Session Replication
Date Thu, 20 Sep 2012 17:43:07 GMT
Hey,

Really hoping somebody can help.  I am attempting a cluster with session
replication.  Tomcat 7, apache with mod_jk.  The cluster and load balancing
seems to be ok but I cannot get session replication working.  I am using
static membership as I am testing this on EC2.  The load balancer and 2
workers are on individual servers.

I have two tomcat instances, tomcatone and tomcattwo.  When starting up
tomcattwo I saw this ONCE on tomcattwo and never again :

*INFO: Manager [localhost#/ClusterApp], requesting session state from
org.apache.catalina.tribes.membership.MemberImpl[tcp://tomcatone:4110,tomcatone,4110,
alive=0, securePort=-1, UDP Port=-1, id={1 2 3 4 5 6 7 8 9 10 11 12 13 14
15 1 }, payload={}, command={}, domain={100 101 108 116 97 45 115 116 97
...(12)}, ]. This operation will timeout if no session state has been
received within 60 seconds.*
*Sep 20, 2012 3:19:45 PM org.apache.catalina.ha.session.DeltaManager
waitForSendAllSessions*
*SEVERE: Manager [localhost#/ClusterApp]: No session state send at 9/20/12
3:18 PM received, timing out after 60,109 ms.*

The failure was due to firewall I think.  Now the firewall is open and I
haven't seen this again.  Weird.  Right now it doesn't look like the nodes
are trying to communicate at all.

Many thanks in advance!

Server.xml's and logs :

*tomcatone server.xml*
*
*
<?xml version='1.0' encoding='utf-8'?>
<!--
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
-->
<!-- Note:  A "Server" is not itself a "Container", so you may not
     define subcomponents such as "Valves" at this level.
     Documentation at /docs/config/server.html
 -->
<Server port="8005" shutdown="SHUTDOWN">
  <!-- Security listener. Documentation at /docs/config/listeners.html
  <Listener className="org.apache.catalina.security.SecurityListener" />
  -->
  <!--APR library loader. Documentation at /docs/apr.html -->
  <Listener className="org.apache.catalina.core.AprLifecycleListener"
SSLEngine="on" />
  <!--Initialize Jasper prior to webapps are loaded. Documentation at
/docs/jasper-howto.html -->
  <Listener className="org.apache.catalina.core.JasperListener" />
  <!-- Prevent memory leaks due to use of particular java/javax APIs-->
  <Listener
className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
  <Listener
className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
  <Listener
className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />

  <!-- Global JNDI resources
       Documentation at /docs/jndi-resources-howto.html
  -->
  <GlobalNamingResources>
    <!-- Editable user database that can also be used by
         UserDatabaseRealm to authenticate users
    -->
    <Resource name="UserDatabase" auth="Container"
              type="org.apache.catalina.UserDatabase"
              description="User database that can be updated and saved"
              factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
              pathname="conf/tomcat-users.xml" />
  </GlobalNamingResources>

  <!-- A "Service" is a collection of one or more "Connectors" that share
       a single "Container" Note:  A "Service" is not itself a "Container",
       so you may not define subcomponents such as "Valves" at this level.
       Documentation at /docs/config/service.html
   -->
  <Service name="Catalina">

    <!--The connectors can use a shared executor, you can define one or
more named thread pools-->
    <!--
    <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
        maxThreads="150" minSpareThreads="4"/>
    -->


    <!-- A "Connector" represents an endpoint by which requests are received
         and responses are returned. Documentation at :
         Java HTTP Connector: /docs/config/http.html (blocking &
non-blocking)
         Java AJP  Connector: /docs/config/ajp.html
         APR (HTTP/AJP) Connector: /docs/apr.html
         Define a non-SSL HTTP/1.1 Connector on port 8080
    -->
    <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
    <!-- A "Connector" using the shared thread pool-->
    <!--
    <Connector executor="tomcatThreadPool"
               port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
    -->
    <!-- Define a SSL HTTP/1.1 Connector on port 8443
         This connector uses the JSSE configuration, when using APR, the
         connector should be using the OpenSSL style configuration
         described in the APR documentation -->
    <!--
    <Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
               maxThreads="150" scheme="https" secure="true"
               clientAuth="false" sslProtocol="TLS" />
    -->

    <!-- Define an AJP 1.3 Connector on port 8009 -->
    <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />


    <!-- An Engine represents the entry point (within Catalina) that
processes
         every request.  The Engine implementation for Tomcat stand alone
         analyzes the HTTP headers included with the request, and passes
them
         on to the appropriate Host (virtual host).
         Documentation at /docs/config/engine.html -->

    <!-- You should set jvmRoute to support load-balancing via AJP ie :
    <Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">
    -->
    <Engine name="Catalina" defaultHost="localhost" jvmRoute="worker1">

      <!--For clustering, please take a look at documentation at:
          /docs/cluster-howto.html  (simple how to)
          /docs/config/cluster.html (reference documentation) -->

      <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="8" channelStartOptions="3">

        <Manager className="org.apache.catalina.ha.session.DeltaManager"

          expireSessionsOnShutdown="false"

          notifyListenersOnReplication="true"/>


        <Channel className="org.apache.catalina.tribes.group.GroupChannel">

                <Receiver
className="org.apache.catalina.tribes.transport.nio.NioReceiver"

                        address="localhost" port="4110" autoBind="100"

                        selectorTimeout="5000" maxThreads="6"/>


                <Sender
className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
                        <Transport
className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
                </Sender>


                <Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpPingInterceptor"/>
                <Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
                <Interceptor
className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>

                <Interceptor
className="org.apache.catalina.tribes.group.interceptors.StaticMembershipInterceptor">
                        <!--Member
className="org.apache.catalina.tribes.membership.StaticMember"
                        port="4110"
                        host="localhost"
                        domain="delta-static"
                        uniqueId="{0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}"
                        /-->
                        <Member
className="org.apache.catalina.tribes.membership.StaticMember"


                                port="4210"
                                host="tomcattwo"
                                domain="delta-static"

uniqueId="{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,2}"
                        />
                </Interceptor>

        </Channel>

        <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=""/>

        <Valve
className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>

        <ClusterListener
className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>

        <ClusterListener
className="org.apache.catalina.ha.session.ClusterSessionListener"/>

      </Cluster>


      <!-- Use the LockOutRealm to prevent attempts to guess user passwords
           via a brute-force attack -->
      <Realm className="org.apache.catalina.realm.LockOutRealm">
        <!-- This Realm uses the UserDatabase configured in the global JNDI
             resources under the key "UserDatabase".  Any edits
             that are performed against this UserDatabase are immediately
             available for use by the Realm.  -->
        <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
               resourceName="UserDatabase"/>
      </Realm>

      <Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="true">

        <!-- SingleSignOn valve, share authentication between web
applications
             Documentation at: /docs/config/valve.html -->
        <!--
        <Valve className="org.apache.catalina.authenticator.SingleSignOn" />
        -->

        <!-- Access log processes all example.
             Documentation at: /docs/config/valve.html
             Note: The pattern used is equivalent to using pattern="common"
-->
        <Valve className="org.apache.catalina.valves.AccessLogValve"
directory="logs"
               prefix="localhost_access_log." suffix=".txt"
               pattern="%h %l %u %t &quot;%r&quot; %s %b" />

      </Host>
    </Engine>
  </Service>
</Server>


*tomcat2 server.xml*
<?xml version='1.0' encoding='utf-8'?>
<!--
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
-->
<!-- Note:  A "Server" is not itself a "Container", so you may not
     define subcomponents such as "Valves" at this level.
     Documentation at /docs/config/server.html
 -->
<Server port="8005" shutdown="SHUTDOWN">
  <!-- Security listener. Documentation at /docs/config/listeners.html
  <Listener className="org.apache.catalina.security.SecurityListener" />
  -->
  <!--APR library loader. Documentation at /docs/apr.html -->
  <Listener className="org.apache.catalina.core.AprLifecycleListener"
SSLEngine="on" />
  <!--Initialize Jasper prior to webapps are loaded. Documentation at
/docs/jasper-howto.html -->
  <Listener className="org.apache.catalina.core.JasperListener" />
  <!-- Prevent memory leaks due to use of particular java/javax APIs-->
  <Listener
className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
  <Listener
className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
  <Listener
className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />

  <!-- Global JNDI resources
       Documentation at /docs/jndi-resources-howto.html
  -->
  <GlobalNamingResources>
    <!-- Editable user database that can also be used by
         UserDatabaseRealm to authenticate users
    -->
    <Resource name="UserDatabase" auth="Container"
              type="org.apache.catalina.UserDatabase"
              description="User database that can be updated and saved"
              factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
              pathname="conf/tomcat-users.xml" />
  </GlobalNamingResources>

  <!-- A "Service" is a collection of one or more "Connectors" that share
       a single "Container" Note:  A "Service" is not itself a "Container",
       so you may not define subcomponents such as "Valves" at this level.
       Documentation at /docs/config/service.html
   -->
  <Service name="Catalina">

    <!--The connectors can use a shared executor, you can define one or
more named thread pools-->
    <!--
    <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
        maxThreads="150" minSpareThreads="4"/>
    -->


    <!-- A "Connector" represents an endpoint by which requests are received
         and responses are returned. Documentation at :
         Java HTTP Connector: /docs/config/http.html (blocking &
non-blocking)
         Java AJP  Connector: /docs/config/ajp.html
         APR (HTTP/AJP) Connector: /docs/apr.html
         Define a non-SSL HTTP/1.1 Connector on port 8080
    -->
    <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
    <!-- A "Connector" using the shared thread pool-->
    <!--
    <Connector executor="tomcatThreadPool"
               port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
    -->
    <!-- Define a SSL HTTP/1.1 Connector on port 8443
         This connector uses the JSSE configuration, when using APR, the
         connector should be using the OpenSSL style configuration
         described in the APR documentation -->
    <!--
    <Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
               maxThreads="150" scheme="https" secure="true"
               clientAuth="false" sslProtocol="TLS" />
    -->

    <!-- Define an AJP 1.3 Connector on port 8009 -->
    <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />


    <!-- An Engine represents the entry point (within Catalina) that
processes
         every request.  The Engine implementation for Tomcat stand alone
         analyzes the HTTP headers included with the request, and passes
them
         on to the appropriate Host (virtual host).
         Documentation at /docs/config/engine.html -->

    <!-- You should set jvmRoute to support load-balancing via AJP ie :
    <Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">
    -->
    <Engine name="Catalina" defaultHost="localhost" jvmRoute="worker2">

      <!--For clustering, please take a look at documentation at:
          /docs/cluster-howto.html  (simple how to)
          /docs/config/cluster.html (reference documentation) -->

     <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="8" channelStartOptions="3">

        <Manager className="org.apache.catalina.ha.session.DeltaManager"

          expireSessionsOnShutdown="false"

          notifyListenersOnReplication="true"/>


  <Channel className="org.apache.catalina.tribes.group.GroupChannel">

        <Receiver
className="org.apache.catalina.tribes.transport.nio.NioReceiver"

            address="localhost" port="4210" autoBind="100"

            selectorTimeout="5000" maxThreads="6"/>


    <Sender
className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
      <Transport
className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
    </Sender>


   <Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpPingInterceptor"/>
          <Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
          <Interceptor
className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>

          <Interceptor
className="org.apache.catalina.tribes.group.interceptors.StaticMembershipInterceptor">
                <!--Member
className="org.apache.catalina.tribes.membership.StaticMember"
                  port="4110"
                  host="localhost"
                  domain="delta-static"
                  uniqueId="{0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}"
                /-->
                <Member
className="org.apache.catalina.tribes.membership.StaticMember"
                  port="4110"
                  host="tomcatone"


                  domain="delta-static"
                  uniqueId="{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,1}"
                />
          </Interceptor>

  </Channel>

<Valve className="org.apache.catalina.ha.tcp.ReplicationValve" filter=""/>

<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>

<ClusterListener
className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>

<ClusterListener
className="org.apache.catalina.ha.session.ClusterSessionListener"/>

      </Cluster>


      <!-- Use the LockOutRealm to prevent attempts to guess user passwords
           via a brute-force attack -->
      <Realm className="org.apache.catalina.realm.LockOutRealm">
        <!-- This Realm uses the UserDatabase configured in the global JNDI
             resources under the key "UserDatabase".  Any edits
             that are performed against this UserDatabase are immediately
             available for use by the Realm.  -->
        <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
               resourceName="UserDatabase"/>
      </Realm>

      <Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="true">

        <!-- SingleSignOn valve, share authentication between web
applications
             Documentation at: /docs/config/valve.html -->
        <!--
        <Valve className="org.apache.catalina.authenticator.SingleSignOn" />
        -->

        <!-- Access log processes all example.
             Documentation at: /docs/config/valve.html
             Note: The pattern used is equivalent to using pattern="common"
-->
        <Valve className="org.apache.catalina.valves.AccessLogValve"
directory="logs"
               prefix="localhost_access_log." suffix=".txt"
               pattern="%h %l %u %t &quot;%r&quot; %s %b" />

      </Host>
    </Engine>
  </Service>
</Server>



*Cluster Start Node 1*
*
*
INFO: Cluster is about to start
Sep 20, 2012 5:13:38 PM org.apache.catalina.tribes.transport.ReceiverBase
bind
INFO: Receiver Server Socket bound to:localhost/127.0.0.1:4110
Sep 20, 2012 5:13:39 PM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive
/usr/local/apache-tomcat-7.0.30/webapps/ClusterApp.war
Sep 20, 2012 5:14:49 PM org.apache.catalina.ha.session.DeltaManager
startInternal
INFO: Register manager localhost#/ClusterApp to cluster element Engine with
name Catalina
Sep 20, 2012 5:14:49 PM org.apache.catalina.ha.session.DeltaManager
startInternal
INFO: Starting clustering manager at localhost#/ClusterApp
Sep 20, 2012 5:14:49 PM org.apache.catalina.ha.session.DeltaManager
getAllClusterSessions
INFO: Manager [localhost#/ClusterApp]: skipping state transfer. No members
active in cluster group.
Sep 20, 2012 5:14:49 PM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
/usr/local/apache-tomcat-7.0.30/webapps/examples
Sep 20, 2012 5:14:50 PM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
/usr/local/apache-tomcat-7.0.30/webapps/docs
Sep 20, 2012 5:14:50 PM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
/usr/local/apache-tomcat-7.0.30/webapps/host-manager
Sep 20, 2012 5:14:50 PM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
/usr/local/apache-tomcat-7.0.30/webapps/ROOT
Sep 20, 2012 5:14:50 PM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
/usr/local/apache-tomcat-7.0.30/webapps/manager
Sep 20, 2012 5:14:51 PM org.apache.catalina.ha.session.JvmRouteBinderValve
startInternal
INFO: JvmRouteBinderValve started
Sep 20, 2012 5:14:51 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-8080"]
Sep 20, 2012 5:14:51 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["ajp-bio-8009"]
Sep 20, 2012 5:14:52 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 73236 ms

*
*


*Cluster Start Node 2*
*
*
INFO: Cluster is about to start
Sep 20, 2012 5:16:45 PM org.apache.catalina.tribes.transport.ReceiverBase
bind
INFO: Receiver Server Socket bound to:localhost/127.0.0.1:4210
Sep 20, 2012 5:16:45 PM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive
/usr/local/apache-tomcat-7.0.30/webapps/ClusterApp.war
Sep 20, 2012 5:17:53 PM org.apache.catalina.ha.session.DeltaManager
startInternal
INFO: Register manager localhost#/ClusterApp to cluster element Engine with
name Catalina
Sep 20, 2012 5:17:53 PM org.apache.catalina.ha.session.DeltaManager
startInternal
INFO: Starting clustering manager at localhost#/ClusterApp
Sep 20, 2012 5:17:53 PM org.apache.catalina.ha.session.DeltaManager
getAllClusterSessions
*INFO: Manager [localhost#/ClusterApp]: skipping state transfer. No members
active in cluster group.*
Sep 20, 2012 5:17:53 PM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
/usr/local/apache-tomcat-7.0.30/webapps/examples
Sep 20, 2012 5:17:54 PM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
/usr/local/apache-tomcat-7.0.30/webapps/docs
Sep 20, 2012 5:17:54 PM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
/usr/local/apache-tomcat-7.0.30/webapps/host-manager
Sep 20, 2012 5:17:55 PM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
/usr/local/apache-tomcat-7.0.30/webapps/ROOT
Sep 20, 2012 5:17:55 PM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
/usr/local/apache-tomcat-7.0.30/webapps/manager
Sep 20, 2012 5:17:55 PM org.apache.catalina.ha.session.JvmRouteBinderValve
startInternal
INFO: JvmRouteBinderValve started
Sep 20, 2012 5:17:55 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-8080"]
Sep 20, 2012 5:17:55 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["ajp-bio-8009"]
Sep 20, 2012 5:17:56 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 70803 ms




-- 
John Rellis

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message