tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From André Warnier (tomcat) ...@ice-sa.com>
Subject Re: Operation has timed out
Date Tue, 07 Feb 2017 17:09:13 GMT
On 07.02.2017 17:20, Fady Haikal wrote:
> Christopher,
> For the first time

@Christopher : just to make sure you got that bit, buried below : the actual replication 
seems to work fine. The problem is only these "unsuccesful ping" messages in the log, 
which fill the log, and which so far nobody has managed to find an explanation for.

>
> On Tue, Feb 7, 2017 at 6:19 PM, Christopher Schultz
> <chris@christopherschultz.net> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA256
>>
>> Fady,
>>
>> On 2/7/17 10:53 AM, Fady Haikal wrote:
>>> Process        PID   Protocol   local address  local port
>>> Remote Address                      State Tomcat8.exe 8160 TCP
>>> imal14-app2    4000 imal14-app1.albaraka.com.sd    ESTABLISHED
>>
>> Stupid question: was this working in the past, and it stopped working?
>> Or are you trying to get this working for the first time?
>>
>> - -chris
>>
>>> On Tue, Feb 7, 2017 at 5:46 PM, Fady Haikal <fadyhaikal@gmail.com>
>>> wrote:
>>>> Yes there is a ESTABLISHED connection, the replication of
>>>> sessions is working fine (port 4000 is for tomcat cluster) but we
>>>> also faced this error on the log file
>>>>
>>>> On Tue, Feb 7, 2017 at 5:44 PM, André Warnier (tomcat)
>>>> <aw@ice-sa.com> wrote:
>>>>> On 07.02.2017 16:24, Fady Haikal wrote:
>>>>>>
>>>>>> Hi, telnet IP 4000 is working fine, i installed a tool for
>>>>>> network monitoring at the level of IP and Port and i didnt
>>>>>> see any disconnection,
>>>>>
>>>>>
>>>>> but did you see a *connection* ? I mean, on the pinging node,
>>>>> if you use the Windows "netstat" program, for example as
>>>>> netstat -aon -p TCP you should see a list of connections in the
>>>>> ESTABLISHED state, of which one of the IP/ports should be your
>>>>> target IP:4000 (in the "remote" column). And on the pinged
>>>>> node, this port :4000 should be in the "local" column, in
>>>>> LISTEN mode (and also probably one in the ESTABLISHED state, if
>>>>> they agree.)
>>>>>
>>>>> Is that the case ?
>>>>>
>>>>>
>>>>>
>>>>> and yes i'm sure that no firewall is enabled.
>>>>>>
>>>>>>
>>>>>> I saw some strange think on the server that I have tried to
>>>>>> ping the multicast IP (228.0.0.4) and i get reply from
>>>>>> different IPs in the network, i don't know why and how i get
>>>>>> those IPs, after checking with the network team they told me
>>>>>> that those IPs are related to the SAN storage taking into
>>>>>> consideration that the Tomcat servers are not connected in
>>>>>> anyway to that SUN storage.
>>>>>>
>>>>>>
>>>>>> On Tue, Feb 7, 2017 at 4:51 PM, André Warnier (tomcat)
>>>>>> <aw@ice-sa.com> wrote:
>>>>>>>
>>>>>>> Hi.
>>>>>>>
>>>>>>> This is for the Tomcat/Tribes experts on the list.
>>>>>>>
>>>>>>> I know nothing of Tribes, but the on-line documentation
>>>>>>> seems to say that the communication happens over TCP and
>>>>>>> that the protocol used is not encrypted. Fady previously
>>>>>>> tried a standard "ping" and a "telnet" between the two
>>>>>>> nodes, and that is the base for him mentioning that "there
>>>>>>> is no network disconnection" between the nodes.
>>>>>>> Nevertheless, the calling pinging node seems to say that it
>>>>>>> times out without getting a response fom the target node.
>>>>>>> There is evidently a contradiction there. So this could
>>>>>>> still be some kind of network issue.
>>>>>>>
>>>>>>> Considering that the protocol command for this "ping"
>>>>>>> should be known by someone here, would it not be possible
>>>>>>> to imagine a little program in some scripting language (or
>>>>>>> even java, God forbid), which would open a TCP channel with
>>>>>>> the target node IP/port, send such a "ping" message, wait
>>>>>>> for a response and report the result ? That would at least
>>>>>>> confirm/deny that the problem is with the network.
>>>>>>>
>>>>>>> The log below does not for example say if the error happens
>>>>>>> when opening the TCP communication channel, or after
>>>>>>> sending the ping message on it, (Of course, testing the TCP
>>>>>>> open could be done with "telnet IP 4000", but I don't know
>>>>>>> if Fady tried this). Maybe tribes also already contains
>>>>>>> some löw-level debugging options ? wireshark maybe another
>>>>>>> option, but it has quite a learning curve. And this is on
>>>>>>> Windows.
>>>>>>>
>>>>>>> By the way Fady, are you sure that your "Windows Firewall
>>>>>>> with Enhanced Security" is not just dropping TCP packets
>>>>>>> to/from port 40xx (or from "java.exe") ? There are some
>>>>>>> "network policies" there which can have wide-ranging
>>>>>>> side-effects.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 07.02.2017 14:42, Fady Haikal wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> Hi, issue still not fixed. Tomcat session replication is
>>>>>>>> not able to replicate the key from node to node, please
>>>>>>>> find below the error, taking into consideration that
>>>>>>>> there is no network disconnection between 2 nodes
>>>>>>>>
>>>>>>>>
>>>>>>>> 07-Feb-2017 16:36:06.186 SEVERE [http-nio-8080-exec-8]
>>>>>>>> org.apache.catalina.tribes.tipis.LazyReplicatedMap.publishEntryIn
>> fo
>>>>>>>>
>>>>>>>>
>> Unable to replicate backup
>>>>>>>> key:58291D242C742A8A4B1657BA42C831A4.TomcatNode2 to
>>>>>>>> backup:org.apache.catalina.tribes.membership.MemberImpl[tcp://{10
>> ,
>>>>>>>>
>>>>>>>>
>> 114, 43, 102}:4000,{10, 114, 43, 102},4000, alive=68841350,
>>>>>>>> securePort=-1, UDP Port=-1, id={85 5 -62 -66 106 -12 64
>>>>>>>> 12 -102 -14 -85 -87 15 9 -51 -112 }, payload={},
>>>>>>>> command={}, domain={}, ]. Reason:Operation has timed
>>>>>>>> out(3000 ms.).; Faulty members:tcp://{10, 114, 43,
>>>>>>>> 102}:4000; org.apache.catalina.tribes.ChannelException:
>>>>>>>> Operation has timed out(3000 ms.).; Faulty
>>>>>>>> members:tcp://{10, 114, 43, 102}:4000; at
>>>>>>>>
>>>>>>>> org.apache.catalina.tribes.transport.nio.ParallelNioSender.sendMe
>> ssage(ParallelNioSender.java:108)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.tribes.transport.nio.PooledParallelSender.sen
>> dMessage(PooledParallelSender.java:48)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.tribes.transport.ReplicationTransmitter.sendM
>> essage(ReplicationTransmitter.java:54)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.tribes.group.ChannelCoordinator.sendMessage(C
>> hannelCoordinator.java:82)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessa
>> ge(ChannelInterceptorBase.java:76)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.tribes.group.interceptors.MessageDispatchInte
>> rceptor.sendMessage(MessageDispatchInterceptor.java:81)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessa
>> ge(ChannelInterceptorBase.java:76)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.
>> sendMessage(TcpFailureDetector.java:93)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessa
>> ge(ChannelInterceptorBase.java:76)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.tribes.group.GroupChannel.send(GroupChannel.j
>> ava:233)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.tribes.group.GroupChannel.send(GroupChannel.j
>> ava:186)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.tribes.tipis.LazyReplicatedMap.publishEntryIn
>> fo(LazyReplicatedMap.java:170)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.tribes.tipis.AbstractReplicatedMap.put(Abstra
>> ctReplicatedMap.java:1040)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.tribes.tipis.AbstractReplicatedMap.put(Abstra
>> ctReplicatedMap.java:1024)
>>>>>>>>
>>>>>>>>
>> at org.apache.catalina.session.ManagerBase.add(ManagerBase.java:647)
>>>>>>>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.session.StandardSession.setId(StandardSession
>> .java:374)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>> org.apache.catalina.ha.session.DeltaSession.setId(DeltaSession.ja
>> va:279)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.session.ManagerBase.createSession(ManagerBase
>> .java:708)
>>>>>>>>
>>>>>>>>
>> at org.apache.catalina.connector.Request.doGetSession(Request.java:2936)
>>>>>>>> at
>>>>>>>> org.apache.catalina.connector.Request.getSession(Request.java:226
>> 0)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.connector.RequestFacade.getSession(RequestFac
>> ade.java:895)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServl
>> etRequestWrapper.java:231)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.core.ApplicationHttpRequest.getSession(Applic
>> ationHttpRequest.java:568)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.core.ApplicationHttpRequest.getSession(Applic
>> ationHttpRequest.java:513)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.jasper.runtime.PageContextImpl.initialize(PageContextI
>> mpl.java:137)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.jasper.runtime.JspFactoryImpl.internalGetPageContext(J
>> spFactoryImpl.java:109)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.jasper.runtime.JspFactoryImpl.getPageContext(JspFactor
>> yImpl.java:60)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>> org.apache.jsp.WEB_002dINF.jsp._401_jsp._jspService(_401_jsp.java
>> :100)
>>>>>>>>
>>>>>>>>
>> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
>>>>>>>> at
>>>>>>>> javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWra
>> pper.java:438)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.ja
>> va:396)
>>>>>>>>
>>>>>>>>
>> at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:340)
>>>>>>>> at
>>>>>>>> javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
>> ApplicationFilterChain.java:291)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(Applicat
>> ionFilterChain.java:206)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.core.ApplicationDispatcher.invoke(Application
>> Dispatcher.java:719)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.core.ApplicationDispatcher.processRequest(App
>> licationDispatcher.java:467)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.core.ApplicationDispatcher.doForward(Applicat
>> ionDispatcher.java:390)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.core.ApplicationDispatcher.forward(Applicatio
>> nDispatcher.java:317)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.core.StandardHostValve.custom(StandardHostVal
>> ve.java:445)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.core.StandardHostValve.status(StandardHostVal
>> ve.java:304)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostVal
>> ve.java:181)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.ha.session.JvmRouteBinderValve.invoke(JvmRout
>> eBinderValve.java:194)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.ha.tcp.ReplicationValve.invoke(ReplicationVal
>> ve.java:318)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportVal
>> ve.java:79)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.valves.StuckThreadDetectionValve.invoke(Stuck
>> ThreadDetectionValve.java:208)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.valves.AbstractAccessLogValve.invoke(Abstract
>> AccessLogValve.java:616)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngin
>> eValve.java:88)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter
>> .java:521)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.coyote.http11.AbstractHttp11Processor.process(Abstract
>> Http11Processor.java:1096)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.proc
>> ess(AbstractProtocol.java:674)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioE
>> ndpoint.java:1500)
>>>>>>>>
>>>>>>>>
>> at
>>>>>>>>
>>>>>>>> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEnd
>> point.java:1456)
>>>>>>>>
>>>>>>>>
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
>>>>>>>> at
>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
>>>>>>>> Source) at
>>>>>>>>
>>>>>>>> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(Ta
>> skThread.java:61)
>>>>>>>>
>>>>>>>>
>> at java.lang.Thread.run(Unknown Source)
>>>>>>>>
>>>>>>>> On Tue, Feb 7, 2017 at 1:02 PM, Kreuser, Peter
>>>>>>>> <pkreuser@airplus.com> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Fady,
>>>>>>>>>
>>>>>>>>> Sorry for top posting.
>>>>>>>>>
>>>>>>>>> If I remember correctly, the Cluster Element goes into
>>>>>>>>> the Container and not the Host. Plus I see in our
>>>>>>>>> (working) case, a DeltaManager and a
>>>>>>>>> JvmRouteSessionIDBinderListener
>>>>>>>>>
>>>>>>>>> <Manager
>>>>>>>>> className="org.apache.catalina.ha.session.DeltaManager"
>>>>>>>>>
>>>>>>>>>
>> expireSessionsOnShutdown="false"
>>>>>>>>> notifyListenersOnReplication="true"/> ...
>>>>>>>>> <ClusterListener
>>>>>>>>>
>>>>>>>>> className="org.apache.catalina.ha.session.JvmRouteSessionIDBinde
>> rListener"/>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>> Besides this, only ports, limits and values are different.
>>>>>>>>>
>>>>>>>>> You may want to filter out the replication for static
>>>>>>>>> resources as gifs jpg or css. <Valve
>>>>>>>>> className="org.apache.catalina.ha.tcp.ReplicationValve"
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>> filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.htm;.*\.html;.*\.css;.*\.txt;
>> " />
>>>>>>>>>
>>>>>>>>> Best regards
>>>>>>>>>
>>>>>>>>> Peter
>>>>>>>>>
>>>>>>>>>> below is the server.xml configuration, as mentioened
>>>>>>>>>> earlier the issue is related to the cluster
>>>>>>>>>> configuration, and as per my research i can see that
>>>>>>>>>> some users are facing the same issue but i didnt
>>>>>>>>>> found the solution of it
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> <?xml version='1.0' encoding='utf-8'?> <!-- Licensed
>>>>>>>>>> to the Apache Software Foundation (ASF) under one or
>>>>>>>>>> more contributor license agreements.  See the NOTICE
>>>>>>>>>> file distributed with this work for additional
>>>>>>>>>> information regarding copyright ownership. The ASF
>>>>>>>>>> licenses this file to You under the Apache License,
>>>>>>>>>> Version 2.0 (the "License"); you may not use this
>>>>>>>>>> file except in compliance with the License.  You may
>>>>>>>>>> obtain a copy of the License at
>>>>>>>>>>
>>>>>>>>>> http://www.apache.org/licenses/LICENSE-2.0
>>>>>>>>>>
>>>>>>>>>> Unless required by applicable law or agreed to in
>>>>>>>>>> writing, software distributed under the License is
>>>>>>>>>> distributed on an "AS IS" BASIS, WITHOUT WARRANTIES
>>>>>>>>>> OR CONDITIONS OF ANY KIND, either express or
>>>>>>>>>> implied. See the License for the specific language
>>>>>>>>>> governing permissions and limitations under the
>>>>>>>>>> License. --> <!-- Note:  A "Server" is not itself a
>>>>>>>>>> "Container", so you may not define subcomponents such
>>>>>>>>>> as "Valves" at this level. Documentation at
>>>>>>>>>> /docs/config/server.html --> <Server port="8005"
>>>>>>>>>> shutdown="SHUTDOWN"> <Listener
>>>>>>>>>> className="org.apache.catalina.startup.VersionLoggerListener"
>>>>>>>>>> /> <!-- Security listener. Documentation at
>>>>>>>>>> /docs/config/listeners.html <Listener
>>>>>>>>>> className="org.apache.catalina.security.SecurityListener"
>>>>>>>>>>
>>>>>>>>>>
>> />
>>>>>>>>>> --> <!--APR library loader. Documentation at
>>>>>>>>>> /docs/apr.html --> <Listener
>>>>>>>>>> className="org.apache.catalina.core.AprLifecycleListener"
>>>>>>>>>>
>>>>>>>>>>
>> SSLEngine="on" />
>>>>>>>>>> <!-- Prevent memory leaks due to use of particular
>>>>>>>>>> java/javax APIs--> <Listener
>>>>>>>>>> className="org.apache.catalina.core.JreMemoryLeakPreventionList
>> ener"
>>>>>>>>>>
>>>>>>>>>>
>> />
>>>>>>>>>> <Listener
>>>>>>>>>>
>>>>>>>>>> className="org.apache.catalina.mbeans.GlobalResourcesLifecycleL
>> istener"
>>>>>>>>>>
>>>>>>>>>>
>> />
>>>>>>>>>> <Listener
>>>>>>>>>> className="org.apache.catalina.core.ThreadLocalLeakPreventionLi
>> stener"
>>>>>>>>>>
>>>>>>>>>>
>> />
>>>>>>>>>>
>>>>>>>>>> <!-- Global JNDI resources Documentation at
>>>>>>>>>> /docs/jndi-resources-howto.html -->
>>>>>>>>>> <GlobalNamingResources> <!-- Editable user database
>>>>>>>>>> that can also be used by UserDatabaseRealm to
>>>>>>>>>> authenticate users --> <Resource name="UserDatabase"
>>>>>>>>>> auth="Container"
>>>>>>>>>> type="org.apache.catalina.UserDatabase"
>>>>>>>>>> description="User database that can be updated and
>>>>>>>>>> saved"
>>>>>>>>>>
>>>>>>>>>> factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
>>>>>>>>>>
>>>>>>>>>>
>> pathname="conf/tomcat-users.xml" />
>>>>>>>>>> </GlobalNamingResources>
>>>>>>>>>>
>>>>>>>>>> <!-- A "Service" is a collection of one or more
>>>>>>>>>> "Connectors" that share a single "Container" Note:  A
>>>>>>>>>> "Service" is not itself a "Container", so you may not
>>>>>>>>>> define subcomponents such as "Valves" at this level.
>>>>>>>>>> Documentation at /docs/config/service.html -->
>>>>>>>>>> <Service name="Catalina">
>>>>>>>>>>
>>>>>>>>>> <!--The connectors can use a shared executor, you can
>>>>>>>>>> define one or more named thread pools--> <!--
>>>>>>>>>> <Executor name="tomcatThreadPool"
>>>>>>>>>> namePrefix="catalina-exec-" maxThreads="150"
>>>>>>>>>> minSpareThreads="4"/> -->
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> <!-- A "Connector" represents an endpoint by which
>>>>>>>>>> requests are received and responses are returned.
>>>>>>>>>> Documentation at : Java HTTP Connector:
>>>>>>>>>> /docs/config/http.html (blocking & non-blocking) Java
>>>>>>>>>> AJP  Connector: /docs/config/ajp.html APR (HTTP/AJP)
>>>>>>>>>> Connector: /docs/apr.html Define a non-SSL/TLS
>>>>>>>>>> HTTP/1.1 Connector on port 8080 --> <Connector
>>>>>>>>>> port="8080" protocol="HTTP/1.1"
>>>>>>>>>> connectionTimeout="60000" maxThreads="500"
>>>>>>>>>> minSpareThreads="25" maxSpareThreads="75"
>>>>>>>>>> enableLookups="false" disableUploadTimeout="true"
>>>>>>>>>> acceptCount="100" redirectPort="8443" /> <!-- A
>>>>>>>>>> "Connector" using the shared thread pool--> <!--
>>>>>>>>>> <Connector executor="tomcatThreadPool" port="8080"
>>>>>>>>>> protocol="HTTP/1.1" connectionTimeout="60000"
>>>>>>>>>> redirectPort="8443" /> --> <!-- Define a SSL/TLS
>>>>>>>>>> HTTP/1.1 Connector on port 8443 This connector uses
>>>>>>>>>> the NIO implementation that requires the JSSE style
>>>>>>>>>> configuration. When using the APR/native
>>>>>>>>>> implementation, the OpenSSL style configuration is
>>>>>>>>>> required as described in the APR/native documentation
>>>>>>>>>> --> <!-- <Connector port="8443"
>>>>>>>>>> protocol="org.apache.coyote.http11.Http11NioProtocol"
>>>>>>>>>>
>>>>>>>>>>
>> maxThreads="150" SSLEnabled="true" scheme="https"
>>>>>>>>>> secure="true" clientAuth="false" sslProtocol="TLS"
>>>>>>>>>> /> -->
>>>>>>>>>>
>>>>>>>>>> <!-- Define an AJP 1.3 Connector on port 8009 -->
>>>>>>>>>> <Connector port="8009" protocol="AJP/1.3"
>>>>>>>>>> redirectPort="8443" />
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> <!-- An Engine represents the entry point (within
>>>>>>>>>> Catalina) that processes every request.  The Engine
>>>>>>>>>> implementation for Tomcat stand alone analyzes the
>>>>>>>>>> HTTP headers included with the request, and passes
>>>>>>>>>> them on to the appropriate Host (virtual host).
>>>>>>>>>> Documentation at /docs/config/engine.html -->
>>>>>>>>>>
>>>>>>>>>> <!-- You should set jvmRoute to support
>>>>>>>>>> load-balancing via AJP ie :
>>>>>>>>>>
>>>>>>>>>> --> <Engine name="Catalina" defaultHost="localhost"
>>>>>>>>>> jvmRoute="TomcatNode1"> <!-- Use the LockOutRealm to
>>>>>>>>>> prevent attempts to guess user passwords via a
>>>>>>>>>> brute-force attack --> <Realm
>>>>>>>>>> className="org.apache.catalina.realm.LockOutRealm">
>>>>>>>>>> <!-- This Realm uses the UserDatabase configured in
>>>>>>>>>> the global JNDI resources under the key
>>>>>>>>>> "UserDatabase".  Any edits that are performed against
>>>>>>>>>> this UserDatabase are immediately available for use
>>>>>>>>>> by the Realm.  --> <Realm
>>>>>>>>>> className="org.apache.catalina.realm.UserDatabaseRealm"
>>>>>>>>>>
>>>>>>>>>>
>> resourceName="UserDatabase"/>
>>>>>>>>>> </Realm>
>>>>>>>>>>
>>>>>>>>>> <Host name="localhost"  appBase="webapps"
>>>>>>>>>> unpackWARs="true" autoDeploy="true"
>>>>>>>>>> startStopThreads="0"> <!--For clustering, please take
>>>>>>>>>> a look at documentation at: /docs/cluster-howto.html
>>>>>>>>>> (simple how to) /docs/config/cluster.html (reference
>>>>>>>>>> documentation) --> <!-- <Cluster
>>>>>>>>>> className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
>>>>>>>>>>
>>>>>>>>>>
>> - -->
>>>>>>>>>>
>>>>>>>>>> <Cluster
>>>>>>>>>> className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
>>>>>>>>>>
>>>>>>>>>>
>> channelSendOptions="4">
>>>>>>>>>> <Channel
>>>>>>>>>> className="org.apache.catalina.tribes.group.GroupChannel">
>>>>>>>>>>
>>>>>>>>>>
>> <Membership
>>>>>>>>>> className="org.apache.catalina.tribes.membership.McastService"
>>>>>>>>>>
>>>>>>>>>>
>> address="228.0.0.4"
>>>>>>>>>> port="45564" frequency="500" dropTime="9000"/>
>>>>>>>>>> <Receiver
>>>>>>>>>> className="org.apache.catalina.tribes.transport.nio.NioReceiver
>> "
>>>>>>>>>>
>>>>>>>>>>
>> address="auto"
>>>>>>>>>> port="4000" autoBind="100" selectorTimeout="5000"
>>>>>>>>>> maxThreads="6"/>
>>>>>>>>>>
>>>>>>>>>> <Sender
>>>>>>>>>>
>>>>>>>>>> className="org.apache.catalina.tribes.transport.ReplicationTran
>> smitter">
>>>>>>>>>>
>>>>>>>>>>
>> <Transport
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> className="org.apache.catalina.tribes.transport.nio.PooledParal
>> lelSender"/>
>>>>>>>>>>
>>>>>>>>>>
>> </Sender>
>>>>>>>>>> <Interceptor
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> className="org.apache.catalina.tribes.group.interceptors.TcpFai
>> lureDetector"/>
>>>>>>>>>>
>>>>>>>>>>
>> <Interceptor
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> className="org.apache.catalina.tribes.group.interceptors.Messag
>> eDispatch15Interceptor"/>
>>>>>>>>>>
>>>>>>>>>>
>> </Channel>
>>>>>>>>>>
>>>>>>>>>> <Valve
>>>>>>>>>> className="org.apache.catalina.ha.tcp.ReplicationValve"
>>>>>>>>>>
>>>>>>>>>>
>> filter=""/>
>>>>>>>>>> <Valve
>>>>>>>>>> className="org.apache.catalina.ha.session.JvmRouteBinderValve"/
>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>> <Deployer
>>>>>>>>>> className="org.apache.catalina.ha.deploy.FarmWarDeployer"
>>>>>>>>>>
>>>>>>>>>>
>> tempDir="D:/imaljava/TomcatNode1/tmp/war-temp/"
>>>>>>>>>>
>>>>>>>>>> deployDir="D:/imaljava/TomcatNode1/tmp/war-deploy/"
>>>>>>>>>>
>>>>>>>>>> watchDir="D:/imaljava/TomcatNode1/tmp/war-listen/"
>>>>>>>>>> watchEnabled="false"/>
>>>>>>>>>>
>>>>>>>>>> <ClusterListener
>>>>>>>>>> className="org.apache.catalina.ha.session.ClusterSessionListene
>> r"/>
>>>>>>>>>>
>>>>>>>>>>
>> </Cluster>
>>>>>>>>>> <!-- SingleSignOn valve, share authentication between
>>>>>>>>>> web applications Documentation at:
>>>>>>>>>> /docs/config/valve.html --> <!-- <Valve
>>>>>>>>>> className="org.apache.catalina.authenticator.SingleSignOn"
>>>>>>>>>> /> -->
>>>>>>>>>>
>>>>>>>>>> <!-- Access log processes all example. Documentation
>>>>>>>>>> at: /docs/config/valve.html Note: The pattern used is
>>>>>>>>>> equivalent to using pattern="common" --> <Valve
>>>>>>>>>> className="org.apache.catalina.valves.AccessLogValve"
>>>>>>>>>>
>>>>>>>>>>
>> directory="logs"
>>>>>>>>>> prefix="localhost_access_log" suffix=".txt"
>>>>>>>>>> pattern="%h %l %u %t &quot;%r&quot; %s %b" /> <Valve
>>>>>>>>>> className="org.apache.catalina.valves.StuckThreadDetectionValve
>> "
>>>>>>>>>>
>>>>>>>>>>
>> threshold="900" />
>>>>>>>>>>
>>>>>>>>>> </Host> </Engine> </Service> </Server>
>>>>>>>>>>
>>>>>>>>>> On Mon, Feb 6, 2017 at 6:51 PM, André Warnier
>>>>>>>>>> (tomcat) <aw@ice-sa.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 06.02.2017 17:45, Fady Haikal wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Hi, What is the host OS ? Windows Server 2012
>>>>>>>>>>>> What is the Tomcat version ? Apache
>>>>>>>>>>>> Tomcat/8.0.30
>>>>>>>>>>>>
>>>>>>>>>>>> Is this problem new ? was this working before ?
>>>>>>>>>>>> how long ? Since cluster implementation
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I still don't know tribes, but then my non-educated
>>>>>>>>>>> guess at this point would be that there is
>>>>>>>>>>> something wrong in your configuration. Can you
>>>>>>>>>>> copy/paste it here ? (remove sensible things like
>>>>>>>>>>> passwords, public IP addresses etc..)(but not to
>>>>>>>>>>> the point of making it uncheckable).
>>>>>>>>>>>
>>>>>>>>>>> Then maybe some tribes-specialist can take over ?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Is there actually something listening on that
>>>>>>>>>>>> address/port ? Tomcat cluster
>>>>>>>>>>>>
>>>>>>>>>>>> the Port 4000 is listening and there is no
>>>>>>>>>>>> disconnection between 2 nodes ping and telnet are
>>>>>>>>>>>> OK
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Feb 6, 2017 at 6:42 PM, André Warnier
>>>>>>>>>>>> (tomcat) <aw@ice-sa.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 06.02.2017 17:24, Fady Haikal wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Plz can i get some help here? This issue is
>>>>>>>>>>>>>> still occurring and it's filling the log file
>>>>>>>>>>>>>> in the Production server
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards, Fady
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi. If you want quick answers, you should
>>>>>>>>>>>>> provide more information. What is the host OS
>>>>>>>>>>>>> ? What is the Tomcat version ? Is this problem
>>>>>>>>>>>>> new ? was this working before ? how long ?
>>>>>>>>>>>>>
>>>>>>>>>>>>> I do not know tribes at all, but according to
>>>>>>>>>>>>> the logfile below, it seems that something is
>>>>>>>>>>>>> trying to "ping" the address 10.114.43.103,
>>>>>>>>>>>>> port 4000, and never getting a response (or at
>>>>>>>>>>>>> least not within 3000ms). Is there actually
>>>>>>>>>>>>> something listening on that address/port ? The
>>>>>>>>>>>>> "netstat" command (available both on Linux and
>>>>>>>>>>>>> Windows) can tell you. If there is something
>>>>>>>>>>>>> listening there, can it respond to whatever is
>>>>>>>>>>>>> ping-ing it ? (routing, firewall, ..)
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Feb 6, 2017 at 8:52 AM, Fady Haikal
>>>>>>>>>>>>>> <fadyhaikal@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Guys, we are facing the below errors in
>>>>>>>>>>>>>>> Tomcat cluster, please advise
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 06-Feb-2017 01:14:20.718 SEVERE
>>>>>>>>>>>>>>> [GroupChannel-Heartbeat-1]
>>>>>>>>>>>>>>> org.apache.catalina.tribes.tipis.AbstractReplicatedMap.hea
>> rtbeat
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>> Unable to send AbstractReplicatedMap.ping message
>>>>>>>>>>>>>>> org.apache.catalina.tribes.ChannelException:
>>>>>>>>>>>>>>> Operation has timed out(3000 ms.).; Faulty
>>>>>>>>>>>>>>> members:tcp://{10, 114, 43, 103}:4000; at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> org.apache.catalina.tribes.transport.nio.ParallelNioSender
>> .sendMessage(ParallelNioSender.java:108)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>> at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> org.apache.catalina.tribes.transport.nio.PooledParallelSen
>> der.sendMessage(PooledParallelSender.java:48)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>> at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> org.apache.catalina.tribes.transport.ReplicationTransmitte
>> r.sendMessage(ReplicationTransmitter.java:54)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>> at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> org.apache.catalina.tribes.group.ChannelCoordinator.sendMe
>> ssage(ChannelCoordinator.java:82)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>> at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> org.apache.catalina.tribes.group.ChannelInterceptorBase.se
>> ndMessage(ChannelInterceptorBase.java:76)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>> at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> org.apache.catalina.tribes.group.interceptors.MessageDispa
>> tchInterceptor.sendMessage(MessageDispatchInterceptor.java:81)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>> at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> org.apache.catalina.tribes.group.ChannelInterceptorBase.se
>> ndMessage(ChannelInterceptorBase.java:76)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>> at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> org.apache.catalina.tribes.group.interceptors.TcpFailureDe
>> tector.sendMessage(TcpFailureDetector.java:93)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>> at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> org.apache.catalina.tribes.group.ChannelInterceptorBase.se
>> ndMessage(ChannelInterceptorBase.java:76)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>> at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> org.apache.catalina.tribes.group.GroupChannel.send(GroupCh
>> annel.java:233)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>> at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> org.apache.catalina.tribes.group.GroupChannel.send(GroupCh
>> annel.java:186)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>> at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> org.apache.catalina.tribes.group.RpcChannel.send(RpcChanne
>> l.java:99)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>> at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> org.apache.catalina.tribes.tipis.AbstractReplicatedMap.pin
>> g(AbstractReplicatedMap.java:267)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>> at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> org.apache.catalina.tribes.tipis.AbstractReplicatedMap.hea
>> rtbeat(AbstractReplicatedMap.java:885)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>> at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> org.apache.catalina.tribes.group.GroupChannel.heartbeat(Gr
>> oupChannel.java:161)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>> at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> org.apache.catalina.tribes.group.GroupChannel$HeartbeatThr
>> ead.run(GroupChannel.java:697)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>> 06-Feb-2017 01:20:51.437 SEVERE [NioReceiver]
>>>>>>>>>>>>>>> org.apache.catalina.tribes.transport.nio.NioReceiver.liste
>> n
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>> Unable
>>>>>>>>>>>>>>> to process request in NioReceiver
>>>>>>>>>>>>>>> java.io.IOException: A non-blocking socket
>>>>>>>>>>>>>>> operation could not be completed
>>>>>>>>>>>>>>> immediately at
>>>>>>>>>>>>>>> sun.nio.ch.SocketDispatcher.close0(Native
>>>>>>>>>>>>>>> Method) at
>>>>>>>>>>>>>>> sun.nio.ch.SocketDispatcher.close(Unknown
>>>>>>>>>>>>>>> Source) at
>>>>>>>>>>>>>>> sun.nio.ch.SocketChannelImpl.kill(Unknown
>>>>>>>>>>>>>>> Source) at
>>>>>>>>>>>>>>> sun.nio.ch.WindowsSelectorImpl.implDereg(Unknown
>>>>>>>>>>>>>>> Source) at
>>>>>>>>>>>>>>> sun.nio.ch.SelectorImpl.processDeregisterQueue(Unknown
>>>>>>>>>>>>>>> Source) at
>>>>>>>>>>>>>>> sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown
>>>>>>>>>>>>>>> Source) at
>>>>>>>>>>>>>>> sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown
>>>>>>>>>>>>>>> Source) at
>>>>>>>>>>>>>>> sun.nio.ch.SelectorImpl.select(Unknown
>>>>>>>>>>>>>>> Source) at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> org.apache.catalina.tribes.transport.nio.NioReceiver.liste
>> n(NioReceiver.java:272)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>> at
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> org.apache.catalina.tribes.transport.nio.NioReceiver.run(N
>> ioReceiver.java:425)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>> at java.lang.Thread.run(Unknown Source)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards, Fady
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -----------------------------------------------------------
>> - ----------
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>>>>>> users-help@tomcat.apache.org
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> ------------------------------------------------------------
>> - ---------
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>>>>> users-help@tomcat.apache.org
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> -------------------------------------------------------------
>> - --------
>>>>>>>>>>>>
>>>>>>>>>>>>
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>>>> users-help@tomcat.apache.org
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --------------------------------------------------------------
>> - -------
>>>>>>>>>>>
>>>>>>>>>>>
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>>> users-help@tomcat.apache.org
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ---------------------------------------------------------------
>> - ------
>>>>>>>>>>
>>>>>>>>>>
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>> users-help@tomcat.apache.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>> -----------------------------------------------------------------
>> - ----
>>>>>>>>
>>>>>>>>
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>>> For additional commands, e-mail:
>>>>>>>> users-help@tomcat.apache.org
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------------------------------------------
>> - ---
>>>>>>>
>>>>>>>
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>> For additional commands, e-mail:
>>>>>>> users-help@tomcat.apache.org
>>>>>>>
>>>>>>
>>>>>> -------------------------------------------------------------------
>> - --
>>>>>>
>>>>>>
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>> For additional commands, e-mail:
>>>>>> users-help@tomcat.apache.org
>>>>>>
>>>>>
>>>>>
>>>>> --------------------------------------------------------------------
>> - -
>>>>>
>>>>>
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>
>>>
>>> ---------------------------------------------------------------------
>>>
>>>
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>
>> -----BEGIN PGP SIGNATURE-----
>> Comment: GPGTools - http://gpgtools.org
>> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>>
>> iQIcBAEBCAAGBQJYmfN4AAoJEBzwKT+lPKRYZuIQAI2NXtOMNcNpd88y2rXRZFCK
>> p3YN8OBo07UMkT+N6OgmkCrWBGRqIk9SZSUugnWP7N1n5nOFpaio/I2n/2dGOXDB
>> oYfPA3KVkWaMQenOAujjyodi7Q7ABUurJ07V+D0zfBtPpqE7up9/edl94pq8znuR
>> KGp1eEKeMlMiZAS9vm5/0ifhhMIDGfE/Y3xjqf+Kb5gHXlA5BrOrlo7+UX26uuqr
>> qIIaobTCKYaQvYE5HP+9FyOsXCd1oicKz9JIPAoc8A7Yj//n+5Ogtoczyu+mSzxL
>> X7/JSRdywbyfFo+1bOaecxmx9Bh9UgyziEh2AuXXMGwf17a58ciN8VCqtJcJC3HO
>> rffZtc9sqm/Tyt5tvo5PgPbpTrvsjeZXMrxUx5QuOjWlkoVOPptACB4LATVmXaoM
>> lHGw0OBCQgNViI5LXMs59QSdj9/J7tzN+0AakFmO0GM2fHjiFzgHTLGa5QA9yO7b
>> wIDpCQj3cDFig+4z95ZFHcGTmcUd9u5a6dO/4BPzFvsR/r6kfmH7qV1mwVu1TkEH
>> 4B6EO5AwCwZJNM2asxFktswNoI4Wl2pP7ZjdxW8tIXchJco0iHn/VhnO1kPeNI8x
>> ws6KXUpZqPp4ylLWYKTLHg9fMc9Ffn6u6jR09o9amEIWnSlinA4R+n9VztWiBR45
>> h9q0x3f+aKuQp2IE0UBE
>> =WoKC
>> -----END PGP SIGNATURE-----
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message