Mailing-List: contact tomcat-dev-help@jakarta.apache.org; run by ezmlm
Date: Tue, 26 Aug 2003 23:56:53 -0400
From: Brent Verner <brent@rcfile.org>
To: tomcat-dev@jakarta.apache.org
Subject: problem with start/stop of CoyoteConnector
Message-ID: <20030827035653.GA14289@rcfile.org>
Mime-Version: 1.0
Content-Type: multipart/mixed; boundary="yrj/dFKFPuw6o+aM"
Content-Disposition: inline
User-Agent: Mutt/1.5.4i

--yrj/dFKFPuw6o+aM
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

Overview: CoytoteConnector.start()/stop() are misbehaving on tomcat-4.1.27

  a) the org.apache.coyote.http11.Http11Protocol handler will _not_ 
     start() successfully after I've stop()ped it.
  b) the  org.apache.jk.server.JkCoyoteHandler handler will start() after
     a stop() but after a number of stop()/start() cycles, it fails to
     start successfully.
  
  In both cases, the connector thinks the handler isAvailable(), but 
  there is simply no service being offered on the specified port. nada.

Is this a known issue with the connector/handler interaction?  A quick
perusal of the source did not lead me to a quick answer, and I won't
have any real time to dig into the problem until after the project
is complete (a couple of months away), so I'm asking y'all.


Details:
  I've got apache/mod_jk setup to loadbalance with a pool of tomcats as
shown below.


         (Round Robin DNS)
     _______________________
     A1         A2        A3
    /--\-------/--\------/--\   <-- mod_jk lb worker balancing/session affinity
  T1P  |     T2P  |    T3P  |   <-- primary Tomcat Services
       |          |         | 
      T1S        T2S       T3S  <-- secondary tomcat Services
  
  On each of the T.P (primary) tomcats, I have a Valve that verifies a 
connection to an external data source (proprietary CMS thingie), and when
it fails, it does a CoyoteConnector.stop() for the Jk handler, sends a 
redirect back thru the DNS layer, so it will get another tomcat that has 
a valid connection to the external data source.

   All of this appeared to work as intended, so I wrote a quickie JSP to
let the client to "restart" the downed connectors which were stop()ped
due to the external data source becoming unavailable.  At this point I
noticed that issuing a stop()/start() sequence to the http11 connector
caused the http11 connector to not _really_ restart (and listen for
connections).  Having seen this "issue" is only allowed the Jk connector
to be "managed" via this JSP.  After a handful of stop()/start() cycles
on the Jk handler, I noticed that the Jk connector wasn't really listening
for connections while the CoyoteConnector thought is was "available".

  In addition to this, the misbehaving handlers are not properly 
reinitialized (to listen for conns) even when reloading the context
via the /manager/ interface.
  
  Can anyone offer any possible solution to this?  It is _not_ critical,
as the client can easily restart the tomcats one at a time if this
situation arises in production.  I'm mostly wanting to see this bug
squashed :-)

  FWIW, I've attached the JSP that handles the restarting of downed
services.

cheers.
  b

-- 
"Develop your talent, man, and leave the world something. Records are 
really gifts from people. To think that an artist would love you enough
to share his music with anyone is a beautiful thing."  -- Duane Allman

--yrj/dFKFPuw6o+aM
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="connectors.jsp"

<%@ page import="
  org.apache.catalina.*,
  org.apache.catalina.core.*,
  org.apache.coyote.tomcat4.CoyoteConnector
"%>
<%
StandardServer server = (StandardServer)ServerFactory.getServer();
Service[] services = server.findServices();
%>
<html>
<head>
  <link rel="stylesheet" type="text/css" href="style/admin.css" />
</head>
<body>
<table class="datatable">
  <tr>
    <td class="datatable-header">
     Service
    </td>
    <td class="datatable-header">
     Handler
    </td>
    <td class="datatable-header">
     Port
    </td>
    <td class="datatable-header">
     Start
    </td>
    <td class="datatable-header">
     Stop
    </td>
    <td class="datatable-header">
     Restart
    </td>
  </td>
<%
for( int i = 0; i < services.length; ++i ){
  StandardService service = (StandardService)services[i];
  //try {
  //  service.stop();
  //  service.start();
  //}
  //catch( Exception ex ){
  //
  //}
  Connector[] connectors = service.findConnectors();
  for( int j = 0; j < connectors.length; ++j ){
    try {
      Connector connector_c = connectors[j];
      CoyoteConnector connector = (CoyoteConnector)connector_c;
      if( "org.apache.jk.server.JkCoyoteHandler".equals(connector.getProtocolHandlerClassName()) ){
        if( request.getParameter("cmd") != null ){
          if( service.toString().equals(request.getParameter("service"))
              && connector.toString().equals(request.getParameter("connector"))
          ){
            if( "start".equals(request.getParameter("cmd")) ){
              if( ! connector.isAvailable() ){
                try{
                  connector.start();
                }
                catch( Exception ex ){
                  ex.printStackTrace();
                }
              }
            }
            else if( "stop".equals(request.getParameter("cmd")) ){
              if( connector.isAvailable() ){
                connector.stop();
              }
            }
            else if( "restart".equals(request.getParameter("cmd")) ){
              if( connector.isAvailable() ){
                try {
                  connector.stop();
                }
                catch( Exception ex ){
                  ex.printStackTrace();
                }
              }
              Thread.currentThread().sleep(100); 
              try{
                connector.start();
              }
              catch( Exception ex ){
                ex.printStackTrace();
              }
            }
            response.sendRedirect(request.getRequestURI());
          }
        }
        boolean started = connector.isAvailable();
%>
  <tr>
    <td class="datatable-cell">
      <%= service %>
    </td>
    <td class="datatable-cell">
      <%= connector.getProtocolHandlerClassName() %>
    </td>
    <td class="datatable-cell">
      <%= connector.getPort() %>
<%
  try{
    java.net.Socket sock = new java.net.Socket("localhost",connector.getPort());
    sock.close();
%>
    <img src="dots/greendot.gif" alt="up" title="Service accepted test connection"/>
<%
  }
  catch( Exception ex ){
%>
    <img src="dots/reddot.gif" alt="down" title="Service did not accept test connection"/>
<%
  }
%>
    </td>
    <td class="datatable-cell">
<% if( started ){ 
%>
      <span class="formbutton">started</span>
<% } 
   else{ 
%>
        <b><a class="formbutton" style="color: #090" href="?cmd=start&service=<%= service %>&connector=<%= connector %>">start</a></b>
<%
  }
%>
    </td>
    <td class="datatable-cell">
<% if( ! started ){ 
%>
      <span class="formbutton">stopped</span>
<% } 
   else{ 
%>
       <b><a class="formbutton" style="color: #900" href="?cmd=stop&service=<%= service %>&connector=<%= connector %>">stop</a></b>
<%
  }
%>
    </td>
    <td class="datatable-cell">
        <b><a class="formbutton" href="?cmd=restart&service=<%= service %>&connector=<%= connector %>">restart</a></b>
    </td>
  </tr>
<%
      }
    }
    catch( Exception ex ){
      
    }
  }
}
%>
</table>
</body>
</html>

--yrj/dFKFPuw6o+aM--