Mailing-List: contact users-help@tomcat.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Tomcat Users List" <users@tomcat.apache.org>
Subject: Re: Operation has timed out
To: users@tomcat.apache.org
References: <CACYK17BFCDBDgFqMNdLmShWNuDs=u7477jKAzXms=Y8DVZBs=A@mail.gmail.com>
 <CACYK17DutgUJ-gkkD1ERJgEzBwR9nbK9cRs1hJ6Qu5kw_iW3mg@mail.gmail.com>
 <5898A767.80200@ice-sa.com>
 <CACYK17CTdWHbMSZrqyFam-Bjn_eOHPcPL84pvZvVJfd6MsKZqA@mail.gmail.com>
 <5898A992.9020501@ice-sa.com>
 <CACYK17B8QSEDYmDCitKeL2V0coRSf_GJ3ivRKLZpcxwMQ9ZOgA@mail.gmail.com>
 <3616EBD11557B84EAC23E8E52973859934B27ED4@EX-FRA-01.AirPlus.net>
 <CACYK17DvckmSuFUtfU5qSedpke2rTKDVyLSE0wCFZpYMRQTnYA@mail.gmail.com>
 <5899DF0D.8030605@ice-sa.com>
 <CACYK17DaspvmNk6nOzL_i96uCMhsUBjEzFYBiQ+3ES7mQvh7tg@mail.gmail.com>
From: =?UTF-8?Q?Andr=c3=a9_Warnier_=28tomcat=29?= <aw@ice-sa.com>
Message-ID: <5899EB5E.2020904@ice-sa.com>
Date: Tue, 7 Feb 2017 16:44:30 +0100
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:38.0) Gecko/20100101
 Thunderbird/38.5.0
MIME-Version: 1.0
In-Reply-To: <CACYK17DaspvmNk6nOzL_i96uCMhsUBjEzFYBiQ+3ES7mQvh7tg@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
archived-at: Tue, 07 Feb 2017 15:43:59 -0000

On 07.02.2017 16:24, Fady Haikal wrote:
> Hi,
> telnet IP 4000 is working fine, i installed a tool for network
> monitoring at the level of IP and Port and i didnt see any
> disconnection,

but did you see a *connection* ?
I mean, on the pinging node, if you use the Windows "netstat" program, for example as
netstat -aon -p TCP
you should see a list of connections in the ESTABLISHED state, of which one of the 
IP/ports should be your target IP:4000 (in the "remote" column).
And on the pinged node, this port :4000 should be in the "local" column, in LISTEN mode
(and also probably one in the ESTABLISHED state, if they agree.)

Is that the case ?


and yes i'm sure that no firewall is enabled.
>
> I saw some strange think on the server that I have tried to ping the
> multicast IP (228.0.0.4) and i get reply from different IPs in the
> network, i don't know why and how i get those IPs, after checking with
> the network team they told me that those IPs are related to the SAN
> storage taking into consideration that the Tomcat servers are not
> connected in anyway to that SUN storage.
>
>
> On Tue, Feb 7, 2017 at 4:51 PM, André Warnier (tomcat) <aw@ice-sa.com> wrote:
>> Hi.
>>
>> This is for the Tomcat/Tribes experts on the list.
>>
>> I know nothing of Tribes, but the on-line documentation seems to say that
>> the communication happens over TCP and that the protocol used is not
>> encrypted.
>> Fady previously tried a standard "ping" and a "telnet" between the two
>> nodes, and that is the base for him mentioning that "there is no network
>> disconnection" between the nodes.
>> Nevertheless, the calling pinging node seems to say that it times out
>> without getting a response fom the target node.  There is evidently a
>> contradiction there.
>> So this could still be some kind of network issue.
>>
>> Considering that the protocol command for this "ping" should be known by
>> someone here, would it not be possible to imagine a little program in some
>> scripting language (or even java, God forbid), which would open a TCP
>> channel with the target node IP/port, send such a "ping" message, wait for a
>> response and report the result ?
>> That would at least confirm/deny that the problem is with the network.
>>
>> The log below does not for example say if the error happens when opening the
>> TCP communication channel, or after sending the ping message on it,
>> (Of course, testing the TCP open could be done with "telnet IP 4000", but I
>> don't know if Fady tried this).
>> Maybe tribes also already contains some löw-level debugging options ?
>> wireshark maybe another option, but it has quite a learning curve.
>> And this is on Windows.
>>
>> By the way Fady, are you sure that your "Windows Firewall with Enhanced
>> Security" is not just dropping TCP packets to/from port 40xx (or from
>> "java.exe") ? There are some "network policies" there which can have
>> wide-ranging side-effects.
>>
>>
>>
>>
>> On 07.02.2017 14:42, Fady Haikal wrote:
>>>
>>> Hi, issue still not fixed. Tomcat session replication is not able to
>>> replicate the key from node to node, please find below the error,
>>> taking into consideration that there is no network disconnection
>>> between 2 nodes
>>>
>>>
>>> 07-Feb-2017 16:36:06.186 SEVERE [http-nio-8080-exec-8]
>>> org.apache.catalina.tribes.tipis.LazyReplicatedMap.publishEntryInfo
>>> Unable to replicate backup
>>> key:58291D242C742A8A4B1657BA42C831A4.TomcatNode2 to
>>> backup:org.apache.catalina.tribes.membership.MemberImpl[tcp://{10,
>>> 114, 43, 102}:4000,{10, 114, 43, 102},4000, alive=68841350,
>>> securePort=-1, UDP Port=-1, id={85 5 -62 -66 106 -12 64 12 -102 -14
>>> -85 -87 15 9 -51 -112 }, payload={}, command={}, domain={}, ].
>>> Reason:Operation has timed out(3000 ms.).; Faulty members:tcp://{10,
>>> 114, 43, 102}:4000;
>>>    org.apache.catalina.tribes.ChannelException: Operation has timed
>>> out(3000 ms.).; Faulty members:tcp://{10, 114, 43, 102}:4000;
>>> at
>>> org.apache.catalina.tribes.transport.nio.ParallelNioSender.sendMessage(ParallelNioSender.java:108)
>>> at
>>> org.apache.catalina.tribes.transport.nio.PooledParallelSender.sendMessage(PooledParallelSender.java:48)
>>> at
>>> org.apache.catalina.tribes.transport.ReplicationTransmitter.sendMessage(ReplicationTransmitter.java:54)
>>> at
>>> org.apache.catalina.tribes.group.ChannelCoordinator.sendMessage(ChannelCoordinator.java:82)
>>> at
>>> org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:76)
>>> at
>>> org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor.sendMessage(MessageDispatchInterceptor.java:81)
>>> at
>>> org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:76)
>>> at
>>> org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.sendMessage(TcpFailureDetector.java:93)
>>> at
>>> org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:76)
>>> at
>>> org.apache.catalina.tribes.group.GroupChannel.send(GroupChannel.java:233)
>>> at
>>> org.apache.catalina.tribes.group.GroupChannel.send(GroupChannel.java:186)
>>> at
>>> org.apache.catalina.tribes.tipis.LazyReplicatedMap.publishEntryInfo(LazyReplicatedMap.java:170)
>>> at
>>> org.apache.catalina.tribes.tipis.AbstractReplicatedMap.put(AbstractReplicatedMap.java:1040)
>>> at
>>> org.apache.catalina.tribes.tipis.AbstractReplicatedMap.put(AbstractReplicatedMap.java:1024)
>>> at org.apache.catalina.session.ManagerBase.add(ManagerBase.java:647)
>>> at
>>> org.apache.catalina.session.StandardSession.setId(StandardSession.java:374)
>>> at
>>> org.apache.catalina.ha.session.DeltaSession.setId(DeltaSession.java:279)
>>> at
>>> org.apache.catalina.session.ManagerBase.createSession(ManagerBase.java:708)
>>> at org.apache.catalina.connector.Request.doGetSession(Request.java:2936)
>>> at org.apache.catalina.connector.Request.getSession(Request.java:2260)
>>> at
>>> org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:895)
>>> at
>>> javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:231)
>>> at
>>> org.apache.catalina.core.ApplicationHttpRequest.getSession(ApplicationHttpRequest.java:568)
>>> at
>>> org.apache.catalina.core.ApplicationHttpRequest.getSession(ApplicationHttpRequest.java:513)
>>> at
>>> org.apache.jasper.runtime.PageContextImpl.initialize(PageContextImpl.java:137)
>>> at
>>> org.apache.jasper.runtime.JspFactoryImpl.internalGetPageContext(JspFactoryImpl.java:109)
>>> at
>>> org.apache.jasper.runtime.JspFactoryImpl.getPageContext(JspFactoryImpl.java:60)
>>> at org.apache.jsp.WEB_002dINF.jsp._401_jsp._jspService(_401_jsp.java:100)
>>> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
>>> at
>>> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:438)
>>> at
>>> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:396)
>>> at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:340)
>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
>>> at
>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
>>> at
>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>>> at
>>> org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:719)
>>> at
>>> org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:467)
>>> at
>>> org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:390)
>>> at
>>> org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:317)
>>> at
>>> org.apache.catalina.core.StandardHostValve.custom(StandardHostValve.java:445)
>>> at
>>> org.apache.catalina.core.StandardHostValve.status(StandardHostValve.java:304)
>>> at
>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:181)
>>> at
>>> org.apache.catalina.ha.session.JvmRouteBinderValve.invoke(JvmRouteBinderValve.java:194)
>>> at
>>> org.apache.catalina.ha.tcp.ReplicationValve.invoke(ReplicationValve.java:318)
>>> at
>>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
>>> at
>>> org.apache.catalina.valves.StuckThreadDetectionValve.invoke(StuckThreadDetectionValve.java:208)
>>> at
>>> org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:616)
>>> at
>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
>>> at
>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:521)
>>> at
>>> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1096)
>>> at
>>> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:674)
>>> at
>>> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500)
>>> at
>>> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1456)
>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>>> at
>>> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
>>> at java.lang.Thread.run(Unknown Source)
>>>
>>> On Tue, Feb 7, 2017 at 1:02 PM, Kreuser, Peter <pkreuser@airplus.com>
>>> wrote:
>>>>
>>>> Fady,
>>>>
>>>> Sorry for top posting.
>>>>
>>>> If I remember correctly, the Cluster Element goes into the Container and
>>>> not the Host.
>>>> Plus I see in our (working) case, a DeltaManager and a
>>>> JvmRouteSessionIDBinderListener
>>>>
>>>>           <Manager className="org.apache.catalina.ha.session.DeltaManager"
>>>>                      expireSessionsOnShutdown="false"
>>>>                      notifyListenersOnReplication="true"/>
>>>> ...
>>>>           <ClusterListener
>>>> className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
>>>>
>>>> Besides this, only ports, limits and values are different.
>>>>
>>>> You may want to filter out the replication for static resources as gifs
>>>> jpg or css.
>>>>           <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
>>>> filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.htm;.*\.html;.*\.css;.*\.txt;" />
>>>>
>>>> Best regards
>>>>
>>>> Peter
>>>>
>>>>> below is the server.xml configuration, as mentioened earlier the issue
>>>>> is related to the cluster configuration, and as per my research i can
>>>>> see that some users are facing the same issue but i didnt  found the
>>>>> solution of it
>>>>>
>>>>>
>>>>> <?xml version='1.0' encoding='utf-8'?>
>>>>> <!--
>>>>>     Licensed to the Apache Software Foundation (ASF) under one or more
>>>>>     contributor license agreements.  See the NOTICE file distributed with
>>>>>     this work for additional information regarding copyright ownership.
>>>>>     The ASF licenses this file to You under the Apache License, Version
>>>>> 2.0
>>>>>     (the "License"); you may not use this file except in compliance with
>>>>>     the License.  You may obtain a copy of the License at
>>>>>
>>>>>         http://www.apache.org/licenses/LICENSE-2.0
>>>>>
>>>>>     Unless required by applicable law or agreed to in writing, software
>>>>>     distributed under the License is distributed on an "AS IS" BASIS,
>>>>>     WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
>>>>> implied.
>>>>>     See the License for the specific language governing permissions and
>>>>>     limitations under the License.
>>>>> -->
>>>>> <!-- Note:  A "Server" is not itself a "Container", so you may not
>>>>>        define subcomponents such as "Valves" at this level.
>>>>>        Documentation at /docs/config/server.html
>>>>>    -->
>>>>> <Server port="8005" shutdown="SHUTDOWN">
>>>>>     <Listener
>>>>> className="org.apache.catalina.startup.VersionLoggerListener" />
>>>>>     <!-- Security listener. Documentation at /docs/config/listeners.html
>>>>>     <Listener className="org.apache.catalina.security.SecurityListener"
>>>>> />
>>>>>     -->
>>>>>     <!--APR library loader. Documentation at /docs/apr.html -->
>>>>>     <Listener className="org.apache.catalina.core.AprLifecycleListener"
>>>>> SSLEngine="on" />
>>>>>     <!-- Prevent memory leaks due to use of particular java/javax APIs-->
>>>>>     <Listener
>>>>> className="org.apache.catalina.core.JreMemoryLeakPreventionListener"
>>>>> />
>>>>>     <Listener
>>>>> className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener"
>>>>> />
>>>>>     <Listener
>>>>> className="org.apache.catalina.core.ThreadLocalLeakPreventionListener"
>>>>> />
>>>>>
>>>>>     <!-- Global JNDI resources
>>>>>          Documentation at /docs/jndi-resources-howto.html
>>>>>     -->
>>>>>     <GlobalNamingResources>
>>>>>       <!-- Editable user database that can also be used by
>>>>>            UserDatabaseRealm to authenticate users
>>>>>       -->
>>>>>       <Resource name="UserDatabase" auth="Container"
>>>>>                 type="org.apache.catalina.UserDatabase"
>>>>>                 description="User database that can be updated and saved"
>>>>>
>>>>> factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
>>>>>                 pathname="conf/tomcat-users.xml" />
>>>>>     </GlobalNamingResources>
>>>>>
>>>>>     <!-- A "Service" is a collection of one or more "Connectors" that
>>>>> share
>>>>>          a single "Container" Note:  A "Service" is not itself a
>>>>> "Container",
>>>>>          so you may not define subcomponents such as "Valves" at this
>>>>> level.
>>>>>          Documentation at /docs/config/service.html
>>>>>      -->
>>>>>     <Service name="Catalina">
>>>>>
>>>>>       <!--The connectors can use a shared executor, you can define one
>>>>> or more named thread pools-->
>>>>>       <!--
>>>>>       <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
>>>>>           maxThreads="150" minSpareThreads="4"/>
>>>>>       -->
>>>>>
>>>>>
>>>>>       <!-- A "Connector" represents an endpoint by which requests are
>>>>> received
>>>>>            and responses are returned. Documentation at :
>>>>>            Java HTTP Connector: /docs/config/http.html (blocking &
>>>>> non-blocking)
>>>>>            Java AJP  Connector: /docs/config/ajp.html
>>>>>            APR (HTTP/AJP) Connector: /docs/apr.html
>>>>>            Define a non-SSL/TLS HTTP/1.1 Connector on port 8080
>>>>>       -->
>>>>>       <Connector port="8080" protocol="HTTP/1.1"
>>>>>                  connectionTimeout="60000" maxThreads="500"
>>>>> minSpareThreads="25" maxSpareThreads="75" enableLookups="false"
>>>>> disableUploadTimeout="true" acceptCount="100" redirectPort="8443" />
>>>>>       <!-- A "Connector" using the shared thread pool-->
>>>>>       <!--
>>>>>       <Connector executor="tomcatThreadPool"
>>>>>                  port="8080" protocol="HTTP/1.1"
>>>>>                  connectionTimeout="60000"
>>>>>                  redirectPort="8443" />
>>>>>       -->
>>>>>       <!-- Define a SSL/TLS HTTP/1.1 Connector on port 8443
>>>>>            This connector uses the NIO implementation that requires the
>>>>> JSSE
>>>>>            style configuration. When using the APR/native implementation,
>>>>> the
>>>>>            OpenSSL style configuration is required as described in the
>>>>> APR/native
>>>>>            documentation -->
>>>>>       <!--
>>>>>       <Connector port="8443"
>>>>> protocol="org.apache.coyote.http11.Http11NioProtocol"
>>>>>                  maxThreads="150" SSLEnabled="true" scheme="https"
>>>>> secure="true"
>>>>>                  clientAuth="false" sslProtocol="TLS" />
>>>>>       -->
>>>>>
>>>>>       <!-- Define an AJP 1.3 Connector on port 8009 -->
>>>>>       <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />
>>>>>
>>>>>
>>>>>       <!-- An Engine represents the entry point (within Catalina) that
>>>>> processes
>>>>>            every request.  The Engine implementation for Tomcat stand
>>>>> alone
>>>>>            analyzes the HTTP headers included with the request, and
>>>>> passes them
>>>>>            on to the appropriate Host (virtual host).
>>>>>            Documentation at /docs/config/engine.html -->
>>>>>
>>>>>       <!-- You should set jvmRoute to support load-balancing via AJP ie :
>>>>>
>>>>>       -->
>>>>>       <Engine name="Catalina" defaultHost="localhost"
>>>>> jvmRoute="TomcatNode1">
>>>>>     <!-- Use the LockOutRealm to prevent attempts to guess user passwords
>>>>>              via a brute-force attack -->
>>>>>         <Realm className="org.apache.catalina.realm.LockOutRealm">
>>>>>           <!-- This Realm uses the UserDatabase configured in the global
>>>>> JNDI
>>>>>                resources under the key "UserDatabase".  Any edits
>>>>>                that are performed against this UserDatabase are
>>>>> immediately
>>>>>                available for use by the Realm.  -->
>>>>>           <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
>>>>>                  resourceName="UserDatabase"/>
>>>>>         </Realm>
>>>>>
>>>>>         <Host name="localhost"  appBase="webapps"
>>>>>               unpackWARs="true" autoDeploy="true" startStopThreads="0">
>>>>>         <!--For clustering, please take a look at documentation at:
>>>>>             /docs/cluster-howto.html  (simple how to)
>>>>>             /docs/config/cluster.html (reference documentation) -->
>>>>>         <!--
>>>>>         <Cluster
>>>>> className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
>>>>>         -->
>>>>>
>>>>>    <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
>>>>>                    channelSendOptions="4">
>>>>>             <Channel
>>>>> className="org.apache.catalina.tribes.group.GroupChannel">
>>>>>               <Membership
>>>>> className="org.apache.catalina.tribes.membership.McastService"
>>>>>                           address="228.0.0.4"
>>>>>                           port="45564"
>>>>>                           frequency="500"
>>>>>                           dropTime="9000"/>
>>>>>               <Receiver
>>>>> className="org.apache.catalina.tribes.transport.nio.NioReceiver"
>>>>>                         address="auto"
>>>>>                         port="4000"
>>>>>                         autoBind="100"
>>>>>                         selectorTimeout="5000"
>>>>>                         maxThreads="6"/>
>>>>>
>>>>>               <Sender
>>>>> className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
>>>>>                 <Transport
>>>>>
>>>>> className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
>>>>>               </Sender>
>>>>>               <Interceptor
>>>>>
>>>>> className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
>>>>>               <Interceptor
>>>>>
>>>>> className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
>>>>>             </Channel>
>>>>>
>>>>>             <Valve
>>>>> className="org.apache.catalina.ha.tcp.ReplicationValve"
>>>>>                    filter=""/>
>>>>>             <Valve
>>>>> className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
>>>>>
>>>>>             <Deployer
>>>>> className="org.apache.catalina.ha.deploy.FarmWarDeployer"
>>>>>                       tempDir="D:/imaljava/TomcatNode1/tmp/war-temp/"
>>>>>                       deployDir="D:/imaljava/TomcatNode1/tmp/war-deploy/"
>>>>>                       watchDir="D:/imaljava/TomcatNode1/tmp/war-listen/"
>>>>>                       watchEnabled="false"/>
>>>>>
>>>>>             <ClusterListener
>>>>> className="org.apache.catalina.ha.session.ClusterSessionListener"/>
>>>>>           </Cluster>
>>>>>           <!-- SingleSignOn valve, share authentication between web
>>>>> applications
>>>>>                Documentation at: /docs/config/valve.html -->
>>>>>           <!--
>>>>>           <Valve
>>>>> className="org.apache.catalina.authenticator.SingleSignOn" />
>>>>>           -->
>>>>>
>>>>>           <!-- Access log processes all example.
>>>>>                Documentation at: /docs/config/valve.html
>>>>>                Note: The pattern used is equivalent to using
>>>>> pattern="common" -->
>>>>>           <Valve className="org.apache.catalina.valves.AccessLogValve"
>>>>> directory="logs"
>>>>>                  prefix="localhost_access_log" suffix=".txt"
>>>>>                  pattern="%h %l %u %t &quot;%r&quot; %s %b" />
>>>>> <Valve className="org.apache.catalina.valves.StuckThreadDetectionValve"
>>>>>                    threshold="900" />
>>>>>
>>>>>         </Host>
>>>>>       </Engine>
>>>>>     </Service>
>>>>> </Server>
>>>>>
>>>>> On Mon, Feb 6, 2017 at 6:51 PM, André Warnier (tomcat) <aw@ice-sa.com>
>>>>> wrote:
>>>>>>
>>>>>> On 06.02.2017 17:45, Fady Haikal wrote:
>>>>>>>
>>>>>>>
>>>>>>> Hi,
>>>>>>> What is the host OS ? Windows Server 2012
>>>>>>> What is the Tomcat version ? Apache Tomcat/8.0.30
>>>>>>>
>>>>>>> Is this problem new ? was this working before ? how long ? Since
>>>>>>> cluster implementation
>>>>>>>
>>>>>>
>>>>>> I still don't know tribes, but then my non-educated guess at this point
>>>>>> would be that there is something wrong in your configuration.
>>>>>> Can you copy/paste it here ? (remove sensible things like passwords,
>>>>>> public
>>>>>> IP addresses etc..)(but not to the point of making it uncheckable).
>>>>>>
>>>>>> Then maybe some tribes-specialist can take over ?
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Is there actually something listening on that address/port ? Tomcat
>>>>>>> cluster
>>>>>>>
>>>>>>> the Port 4000 is listening and there is no disconnection between 2
>>>>>>> nodes ping and telnet are OK
>>>>>>>
>>>>>>> On Mon, Feb 6, 2017 at 6:42 PM, André Warnier (tomcat) <aw@ice-sa.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> On 06.02.2017 17:24, Fady Haikal wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Plz can i get some help here?
>>>>>>>>> This issue is still occurring and it's filling the log file in the
>>>>>>>>> Production server
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Fady
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Hi.
>>>>>>>> If you want quick answers, you should provide more information.
>>>>>>>> What is the host OS ?
>>>>>>>> What is the Tomcat version ?
>>>>>>>> Is this problem new ? was this working before ? how long ?
>>>>>>>>
>>>>>>>> I do not know tribes at all, but according to the logfile below, it
>>>>>>>> seems
>>>>>>>> that something is trying to "ping" the address 10.114.43.103, port
>>>>>>>> 4000,
>>>>>>>> and
>>>>>>>> never getting a response (or at least not within 3000ms).
>>>>>>>> Is there actually something listening on that address/port ?
>>>>>>>> The "netstat" command (available both on Linux and Windows) can tell
>>>>>>>> you.
>>>>>>>> If there is something listening there, can it respond to whatever is
>>>>>>>> ping-ing it ?
>>>>>>>> (routing, firewall, ..)
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Feb 6, 2017 at 8:52 AM, Fady Haikal <fadyhaikal@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Guys, we are facing the below errors in Tomcat cluster, please
>>>>>>>>>> advise
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 06-Feb-2017 01:14:20.718 SEVERE [GroupChannel-Heartbeat-1]
>>>>>>>>>> org.apache.catalina.tribes.tipis.AbstractReplicatedMap.heartbeat
>>>>>>>>>> Unable to send AbstractReplicatedMap.ping message
>>>>>>>>>>      org.apache.catalina.tribes.ChannelException: Operation has
>>>>>>>>>> timed
>>>>>>>>>> out(3000 ms.).; Faulty members:tcp://{10, 114, 43, 103}:4000;
>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.catalina.tribes.transport.nio.ParallelNioSender.sendMessage(ParallelNioSender.java:108)
>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.catalina.tribes.transport.nio.PooledParallelSender.sendMessage(PooledParallelSender.java:48)
>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.catalina.tribes.transport.ReplicationTransmitter.sendMessage(ReplicationTransmitter.java:54)
>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.catalina.tribes.group.ChannelCoordinator.sendMessage(ChannelCoordinator.java:82)
>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:76)
>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor.sendMessage(MessageDispatchInterceptor.java:81)
>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:76)
>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.sendMessage(TcpFailureDetector.java:93)
>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:76)
>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.catalina.tribes.group.GroupChannel.send(GroupChannel.java:233)
>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.catalina.tribes.group.GroupChannel.send(GroupChannel.java:186)
>>>>>>>>>> at
>>>>>>>>>> org.apache.catalina.tribes.group.RpcChannel.send(RpcChannel.java:99)
>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.catalina.tribes.tipis.AbstractReplicatedMap.ping(AbstractReplicatedMap.java:267)
>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.catalina.tribes.tipis.AbstractReplicatedMap.heartbeat(AbstractReplicatedMap.java:885)
>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.catalina.tribes.group.GroupChannel.heartbeat(GroupChannel.java:161)
>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.catalina.tribes.group.GroupChannel$HeartbeatThread.run(GroupChannel.java:697)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 06-Feb-2017 01:20:51.437 SEVERE [NioReceiver]
>>>>>>>>>> org.apache.catalina.tribes.transport.nio.NioReceiver.listen Unable
>>>>>>>>>> to
>>>>>>>>>> process request in NioReceiver
>>>>>>>>>>      java.io.IOException: A non-blocking socket operation could not
>>>>>>>>>> be
>>>>>>>>>> completed immediately
>>>>>>>>>> at sun.nio.ch.SocketDispatcher.close0(Native Method)
>>>>>>>>>> at sun.nio.ch.SocketDispatcher.close(Unknown Source)
>>>>>>>>>> at sun.nio.ch.SocketChannelImpl.kill(Unknown Source)
>>>>>>>>>> at sun.nio.ch.WindowsSelectorImpl.implDereg(Unknown Source)
>>>>>>>>>> at sun.nio.ch.SelectorImpl.processDeregisterQueue(Unknown Source)
>>>>>>>>>> at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source)
>>>>>>>>>> at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source)
>>>>>>>>>> at sun.nio.ch.SelectorImpl.select(Unknown Source)
>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.catalina.tribes.transport.nio.NioReceiver.listen(NioReceiver.java:272)
>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.catalina.tribes.transport.nio.NioReceiver.run(NioReceiver.java:425)
>>>>>>>>>> at java.lang.Thread.run(Unknown Source)
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Fady
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>>>>
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>>>
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>>
>>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org