tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Eggers <its_toas...@yahoo.com>
Subject Re: Apache AJP to 4 TCs
Date Thu, 09 Sep 2010 22:46:11 GMT
Weird.

I just updated my environment (on Fedora 13) to support 4 Tomcats and it worked 
as expected. I verified this by running my test application, watching traffic 
via jk-manager, and then shutting down (disabling) one mod_jk connection at a 
time. Clustering worked, farm deployment worked, and the application (such as it 
is) worked.

Here are the particulars:

OS:     Fedora 13
JRE:    1.6.0_21 (32 bit)
Apache: 2.2.16 (comes with OS - not the latest)
Tomcat: 6.0.29
mod_jk: 1.2.30 (compiled locally)

My workers.properties file (without all the comments)

worker.list=jk-status,jk-manager,lb

worker.jk-status.type=status
worker.jk-status.read_only=true

worker.jk-manager.type=status

worker.template.type=ajp13
worker.template.host=192.168.0.254
worker.template.socket_connect_timeout=5000
worker.template.socket_keepalive=true
worker.template.ping_mode=A
worker.template.ping_timeout=10000
worker.template.connection_pool_minsize=0
worker.template.connection_pool_timeout=600
worker.template.reply_timeout=300000
worker.template.recovery_options=3

worker.deimos.reference=worker.template
worker.deimos.port=8019

worker.mars.reference=worker.template
worker.mars.port=8029

worker.phobos.reference=worker.template
worker.phobos.port=8039

worker.xerxes.reference=worker.template
worker.xerxes.port=8049

worker.lb.type=lb
worker.lb.error_escalation_time=0
worker.lb.max_reply_timeouts=10
worker.lb.balance_workers=deimos,mars,phobos,xerxes

My uriworkermap.properties file is next. The RPets application is a random pet 
generator I use for testing clustering.

/examples=lb
/examples/*=lb

/docs=lb
/docs/*=lb

/RPets=lb
/RPets/*=lb

mod_jk is added as an included file to httpd.conf. Here is my configuration.

LoadModule jk_module modules/mod_jk.so
<IfModule jk_module>
    JkWorkersFile conf.d/workers.properties

    JkLogFile logs/mod_jk.log
    JkLogLevel info

    JkShmFile /var/run/httpd/mod_jk.shm

    JkOptions +RejectUnsafeURI

    JkWatchdogInterval 60

    <Location /jk-status>
        JkMount jk-status
        Order deny,allow
        Deny from all
        Allow from 127.0.0.1
    </Location>
    <Location /jk-manager>
        JkMount jk-manager
        Order deny,allow
        Deny from all
        Allow from 127.0.0.1
    </Location>

    JkMountFile conf.d/uriworkermap.properties

</IfModule>

Here is one (of four) server.xml files stripped of comments and some 
miscellaneous global resources. I apologize for any line wrapping. Some of the 
package names are quite long.

<?xml version='1.0' encoding='utf-8'?>
<Server port="8015" shutdown="SHUTDOWN">
  <Listener className="org.apache.catalina.core.AprLifecycleListener"
    SSLEngine="on" />
  <Listener className="org.apache.catalina.core.JasperListener" />
  <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" 
/>
  <Listener className="org.apache.catalina.mbeans.ServerLifecycleListener" />
  <Listener 
className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
  <GlobalNamingResources>
    <Resource name="UserDatabase" auth="Container"
              type="org.apache.catalina.UserDatabase"
              description="User database that can be updated and saved"
              factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
              pathname="conf/tomcat-users.xml" />
  </GlobalNamingResources>
  <Service name="Catalina">
    <Connector port="8090" protocol="HTTP/1.1" 
               connectionTimeout="20000"
               redirectPort="8453"/>
    <Connector port="8019" protocol="AJP/1.3" redirectPort="8453"/>
    <Engine name="Catalina" defaultHost="localhost" jvmRoute="deimos">
      <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
     resourceName="UserDatabase"/>
      <Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="true"
            xmlValidation="false" xmlNamespaceAware="false">
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
 channelSendOptions="6">
          <Manager className="org.apache.catalina.ha.session.DeltaManager"
                   expireSessionsOnShutdown="false"
                   notifyListenersOnReplication="true"/>
          <Channel className="org.apache.catalina.tribes.group.GroupChannel">
            <Membership 
className="org.apache.catalina.tribes.membership.McastService"
                        address="228.0.0.4"
                        port="45564"
                        frequency="500"
                        dropTime="3000"/>
            <Receiver 
className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                      address="auto"
                      port="4000"
                      selectorTimeout="100"
                      maxThreads="6"/>
            <Sender 
className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
              <Transport
               
className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
            </Sender>
            <Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
            <Interceptor
className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>

            <Interceptor
           
className="org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor"/>

          </Channel>
          <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
                 
filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.htm;.*\.html;.*\.css;.*\.txt;"/>
          <Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
                    tempDir="${catalina.base}/temp-dir/"
                    deployDir="${catalina.base}/webapps/"
                    watchDir="${catalina.base}/watch-dir/"
    processDeployFrequency="4"
                    watchEnabled="true"/>
          <ClusterListener
                   
 className="org.apache.catalina.ha.session.ClusterSessionListener"/>
        </Cluster>
        <Valve className="org.apache.catalina.valves.AccessLogValve" 
directory="logs"
               prefix="localhost_access_log." suffix=".txt" pattern="common" 
               resolveHosts="false"/>
      </Host>
    </Engine>
  </Service>
</Server>

It's all pretty generic. This particular instance happens to be the one that 
provides the other three with the web application, so I have 
watcheEnabled="true" for the Deployer. In general here's what you have to watch 
out for when running all of this on one machine.

1. Change ports for everything
   shutdown port
   HTTP port (if used - I use it for management)
   HTTPS port (if used)
   AJP port
   Receiver port in cluster (not strictly necessary since Tomcat will figure 
this out)
2. Match jvmroute with names used in workers.properties
3. Match the AJP ports between server.xml and workers.properties

I run this using $CATALINA_HOME and $CATALINA_BASE (%CATALINA_HOME% and 
%CATALINA_BASE% on Windows) plus cluster-startup.sh and a cluster-shutdown.sh 
scripts.

I haven't tried this on Windows yet, so the issue may be with shared memory 
allocation.

Take a close look at your workers.properties files and compare it to the one 
above (which is based on the stock one provided with the tomcat-connectors 
source package).

Hope this helps.

. . . . just my two cents.

/mde/

----- Original Message ----
From: Christopher Schultz <chris@christopherschultz.net>
To: Tomcat Users List <users@tomcat.apache.org>
Sent: Thu, September 9, 2010 11:15:14 AM
Subject: Re: Apache AJP to 4 TCs

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Andrew,

On 9/9/2010 2:00 AM, Andrew Bruno wrote:
> [Wed Sep 08 22:57:36 2010] [6912:6336] [debug] jk_worker.c (242):
> creating worker loadbalancer
> [Wed Sep 08 22:57:36 2010] [6912:6336] [debug] jk_worker.c (146):
> about to create instance loadbalancer of lb
> [Wed Sep 08 22:57:36 2010] [6912:6336] [debug] jk_worker.c (159):
> about to validate and init loadbalancer
> [Wed Sep 08 22:57:36 2010] [6912:6336] [error] jk_lb_worker.c (1536):
> allocating lb sub worker record from shared memory
> [Wed Sep 08 22:57:36 2010] [6912:6336] [error] jk_worker.c (163):
> validate failed for loadbalancer
> [Wed Sep 08 22:57:36 2010] [6912:6336] [error] jk_worker.c (262):
> failed to create worker loadbalancer

D'oh. In your original post, did you include your entire
workers.properties file? Also, can you post all your Jk* directives you
have active in your httpd configuration?

The code that performs validation of the worker is found in
native/common/jk_lb_worker.c:

static int JK_METHOD validate(jk_worker_t *pThis,
                              jk_map_t *props,
                              jk_worker_env_t *we, jk_logger_t *l)


> [Wed Sep 08 22:57:36 2010] [6912:6336] [error] jk_lb_worker.c (1536):
> allocating lb sub worker record from shared memory

This occurs on line 1536 (in mod_jk 1.2.30), in this neighborhood:

>    1533             for (i = 0; i < num_of_workers; i++) {
>    1534                 p->lb_workers[i].s = 
jk_shm_alloc_lb_sub_worker(&p->p);
>    1535                 if (p->lb_workers[i].s == NULL) {
>    1536                     jk_log(l, JK_LOG_ERROR,
>    1537                            "allocating lb sub worker record from shared 
>memory");
>    1538                     JK_TRACE_EXIT(l);
>    1539                     return JK_FALSE;
>    1540                 }
>    1541             }

This looks like it might be a shared memory allocation error. Once you
get into the jk_shm and jk_pool code, there's no debug logging of any
kind :(

You might have to hack-in some debug logging if Rainer or Mladen can't
comment further. Are you comfortable compiling your own mod_jk? It looks
like compiling with DEBUG #defined to something will at least get you
jk_dump_pool calls when shared memory fails to be allocated.

- -chris


      


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message