tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Filip Hanik - Dev Lists <devli...@hanik.com>
Subject Re: tomcat 6 session replication issues
Date Mon, 06 Apr 2009 07:49:43 GMT
Apr 1, 2009 3:28:42 AM 
org.apache.catalina.tribes.transport.nio.NioReceiver socketTimeouts
WARNING: Channel key is registered, but has had no interest ops for the 
last 3000 ms.

this is a sign of a thread being stuck, and its confirmed by the session 
transfer timing out.
Do a thread dump on the server where you are seeing this message, that 
will tell us where your system is stuck

Filip

Jimmy Phillips wrote:
> Hi,
>
> I've been having issues with tomcat session replication. I have a 
> number of tomcat servers running in a cluster mode, behind an Apache 
> load balancer. The tomcat version is 6.0.18 on CentOS 5.1. Running the 
> cluster using the DeltaManager seems to be working fine, however when 
> I try to use the BackupManager for session replication, I get the 
> following entries in the logs:
>
> Apr 1, 2009 3:28:42 AM 
> org.apache.catalina.tribes.transport.nio.NioReceiver socketTimeouts
> WARNING: Channel key is registered, but has had no interest ops for 
> the last 3000 ms. 
> (cancelled:false):sun.nio.ch.SelectionKeyImpl@62af9d74 last 
> access:2009-04-01 03:28:35.969
> Apr 1, 2009 3:28:42 AM 
> org.apache.catalina.tribes.transport.nio.NioReceiver socketTimeouts
> WARNING: Channel key is registered, but has had no interest ops for 
> the last 3000 ms. 
> (cancelled:false):sun.nio.ch.SelectionKeyImpl@4c4947d3 last 
> access:2009-04-01 03:28:35.969
> Apr 1, 2009 3:29:04 AM 
> org.apache.catalina.tribes.group.interceptors.TcpFailureDetector 
> memberDisappeared
> INFO: Received 
> memberDisappeared[org.apache.catalina.tribes.membership.MemberImpl[tcp://{10, 
> 99, 86, 47}:4000,{10, 99, 86, 47},4000, alive=1380182,id={-121 25 -2 
> -7 81 -1 76 3 -92 -20 122 69 67 102 -31 -15 }, payload={}, command={}, 
> domain={}, ]] message. Will verify.
> Apr 1, 2009 3:29:04 AM 
> org.apache.catalina.tribes.group.interceptors.TcpFailureDetector 
> memberDisappeared
> INFO: Verification complete. Member still 
> alive[org.apache.catalina.tribes.membership.MemberImpl[tcp://{10, 99, 
> 86, 47}:4000,{10, 99, 86, 47},4000, alive=1380182,id={-121 25 -2 -7 81 
> -1 76 3 -92 -20 122 69 67 102 -31 -15 }, payload={}, command={}, 
> domain={}, ]]
> Apr 1, 2009 3:29:04 AM 
> org.apache.catalina.tribes.tipis.AbstractReplicatedMap heartbeat
> SEVERE: Unable to send AbstractReplicatedMap.ping message
> org.apache.catalina.tribes.ChannelException: Operation has timed 
> out(60000 ms.).; Faulty members:tcp://{10, 99, 86, 47}:4000;
>         at 
> org.apache.catalina.tribes.transport.nio.ParallelNioSender.sendMessage(ParallelNioSender.java:97)
>         at 
> org.apache.catalina.tribes.transport.nio.PooledParallelSender.sendMessage(PooledParallelSender.java:53)
>         at 
> org.apache.catalina.tribes.transport.ReplicationTransmitter.sendMessage(ReplicationTransmitter.java:80)
>         at 
> org.apache.catalina.tribes.group.ChannelCoordinator.sendMessage(ChannelCoordinator.java:78)
>         at 
> org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:75)
>         at 
> org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor.sendMessage(ThroughputInterceptor.java:61)
>         at 
> org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:75)
>         at 
> org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor.sendMessage(MessageDispatchInterceptor.java:73)
>         at 
> org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:75)
>         at 
> org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.sendMessage(TcpFailureDetector.java:87)
>         at 
> org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:75)
>         at 
> org.apache.catalina.tribes.group.GroupChannel.send(GroupChannel.java:216)
>         at 
> org.apache.catalina.tribes.group.GroupChannel.send(GroupChannel.java:175)
>         at 
> org.apache.catalina.tribes.group.RpcChannel.send(RpcChannel.java:89)
>         at 
> org.apache.catalina.tribes.tipis.AbstractReplicatedMap.ping(AbstractReplicatedMap.java:253)
>         at 
> org.apache.catalina.tribes.tipis.AbstractReplicatedMap.heartbeat(AbstractReplicatedMap.java:793)
>         at 
> org.apache.catalina.tribes.group.GroupChannel.heartbeat(GroupChannel.java:153)
>         at 
> org.apache.catalina.tribes.group.GroupChannel$HeartbeatThread.run(GroupChannel.java:661)
>
> Of course the above entry is just one of many, for the different 
> hosts. Searching the mailing lists, I found this post 
> http://markmail.org/message/jv4dykh7fdhr4mvp which looks like the same 
> problem I am having. The outcome of that thread states that the 
> problem is fixed by a patch in revision 618823, so I compiled a 
> version of the current 6.x trunk (rev 759722) and deployed it to all 
> the servers. However, the problem is still appearing. I've attached a 
> copy of the current server.xml ( it is common to all tomcat instances ).
>
> I've done a thread dump on one of the servers when these errors 
> started appearing, and the output is attached, thread_dump.txt 
> (removed threads that were running by our application).
>
> This problem is reproducable each time I restart the servers. At this 
> stage, I'm clueless on what to try next, so I'm looking forward to 
> your replies.
>
>
> Regards,
> Jim.
>
> Attached: server.xml, thread_dump.txt
>
> ------------------------------------------------------------------------
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message