tomcat-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Rossbach ...@objektpark.de>
Subject Re: Tomcat 6 - Cluster error.
Date Wed, 19 Dec 2007 12:14:29 GMT
I think your Membership DropTime 4sec is very small. Some more Load  
or a FULL GC can drop your member.
Nomally I use 30 sec.

Peter

Am 19.12.2007 um 11:40 schrieb Raúl García:

> Hi,
>
> We are using tomcat 6.0.14.
>
> We use a cluster working at the same machine,
> we start it and it seems to be ok, replication is working fine.
>
> But within the next 12h from startup, we see this error at the  
> catalina.out
> log file, of the first instance, the error repeats forever until we  
> stop
> both
>
> instances and restart them again.
>
> The server is supporting a 0.5 hit per seccond aprox.
>
> CATALINA.OUT
> ===============
> Dec 19, 2007 10:07:30 AM
> org.apache.catalina.tribes.group.interceptors.TcpFailureDetector
> performBasicCheck
> WARNING: Member added, even though we werent
> notified:org.apache.catalina.tribes.membership.MemberImpl[tcp:// 
> localhost:40
> 02,localhost,4002,
>
> alive=165023279,id={-42 -48 112 17 -57 -2 73 -111 -109 113 -93 84 6  
> 91 -72
> 102 }, payload={}, command={}, domain={}, ]
> Dec 19, 2007 10:07:30 AM org.apache.catalina.ha.tcp.SimpleTcpCluster
> memberAdded
> INFO: Replication member
> added:org.apache.catalina.tribes.membership.MemberImpl[tcp:// 
> localhost:4002,
> localhost,4002, alive=165023279,id={-42 -48 112 17 -57
>
> -2 73 -111 -109 113 -93 84 6 91 -72 102 }, payload={}, command={},
> domain={}, ]
> Dec 19, 2007 10:07:34 AM
> org.apache.catalina.tribes.group.interceptors.TcpFailureDetector
> memberDisappeared
> INFO: Received
> memberDisappeared[org.apache.catalina.tribes.membership.MemberImpl 
> [tcp://loc
> alhost:4002,localhost,4002, alive=165028289,id={-42 -48 112 17 -57
>
> -2 73 -111 -109 113 -93 84 6 91 -72 102 }, payload={}, command={},
> domain={}, ]] message. Will verify.
> Dec 19, 2007 10:07:34 AM
> org.apache.catalina.tribes.group.interceptors.TcpFailureDetector
> memberDisappeared
> INFO: Verification complete. Member still
> alive[org.apache.catalina.tribes.membership.MemberImpl[tcp:// 
> localhost:4002,
> localhost,4002, alive=165028289,id={-42
>
> -48 112 17 -57 -2 73 -111 -109 113 -93 84 6 91 -72 102 }, payload={},
> command={}, domain={}, ]]
> Dec 19, 2007 10:07:34 AM  
> org.apache.catalina.ha.tcp.SimpleTcpCluster send
> SEVERE: Unable to send message through cluster sender.
> org.apache.catalina.tribes.ChannelException: Operation has timed out 
> (3000
> ms.).; Faulty members:tcp://localhost:4002;
>         at
> org.apache.catalina.tribes.transport.nio.ParallelNioSender.sendMessage 
> (Paral
> lelNioSender.java:97)
>         at
> org.apache.catalina.tribes.transport.nio.PooledParallelSender.sendMess 
> age(Po
> oledParallelSender.java:53)
>         at
> org.apache.catalina.tribes.transport.ReplicationTransmitter.sendMessag 
> e(Repl
> icationTransmitter.java:80)
>         at
> org.apache.catalina.tribes.group.ChannelCoordinator.sendMessage 
> (ChannelCoord
> inator.java:78)
>         at
> org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage 
> (ChannelI
> nterceptorBase.java:75)
>         at
> org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor.se 
> ndMess
> age(ThroughputInterceptor.java:61)
>         at
> org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage 
> (ChannelI
> nterceptorBase.java:75)
>         at
> org.apache.catalina.tribes.group.interceptors.MessageDispatchIntercept 
> or.sen
> dMessage(MessageDispatchInterceptor.java:73)
>         at
> org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage 
> (ChannelI
> nterceptorBase.java:75)
>         at
> org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.sendM 
> essage
> (TcpFailureDetector.java:87)
>         at
> org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage 
> (ChannelI
> nterceptorBase.java:75)
>         at
> org.apache.catalina.tribes.group.GroupChannel.send 
> (GroupChannel.java:216)
>         at
> org.apache.catalina.tribes.group.GroupChannel.send 
> (GroupChannel.java:175)
>         at
> org.apache.catalina.ha.tcp.SimpleTcpCluster.send 
> (SimpleTcpCluster.java:835)
>         at
> org.apache.catalina.ha.tcp.SimpleTcpCluster.sendClusterDomain 
> (SimpleTcpClust
> er.java:814)
>         at
> org.apache.catalina.ha.session.DeltaManager.send(DeltaManager.java: 
> 586)
>         at
> org.apache.catalina.ha.session.DeltaManager.sendCreateSession 
> (DeltaManager.j
> ava:575)
>         at
> org.apache.catalina.ha.session.DeltaManager.createSession 
> (DeltaManager.java:
> 551)
>         at
> org.apache.catalina.ha.session.DeltaManager.createSession 
> (DeltaManager.java:
> 534)
>         at
> org.apache.catalina.connector.Request.doGetSession(Request.java:2312)
>         at
> org.apache.catalina.connector.Request.getSession(Request.java:2075)
>         at
> org.apache.catalina.connector.RequestFacade.getSession 
> (RequestFacade.java:83
> 3)
>         at pad.kernel.Resolver.service(Resolver.java:266)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java: 
> 803)
>         at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter 
> (Application
> FilterChain.java:290)
>         at
> org.apache.catalina.core.ApplicationFilterChain.doFilter 
> (ApplicationFilterCh
> ain.java:206)
>         at pad.kernel.EntryPointFilter.doFilter 
> (EntryPointFilter.java:365)
>         at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter 
> (Application
> FilterChain.java:235)
>         at
> org.apache.catalina.core.ApplicationFilterChain.doFilter 
> (ApplicationFilterCh
> ain.java:206)
>         at
> org.apache.catalina.core.StandardWrapperValve.invoke 
> (StandardWrapperValve.ja
> va:219)
>         at
> org.apache.catalina.core.StandardContextValve.invoke 
> (StandardContextValve.ja
> va:175)
>         at
> org.apache.catalina.core.StandardHostValve.invoke 
> (StandardHostValve.java:128
> )
>         at
> org.apache.catalina.valves.ErrorReportValve.invoke 
> (ErrorReportValve.java:102
> )
>         at
> org.apache.catalina.valves.RequestFilterValve.process 
> (RequestFilterValve.jav
> a:269)
>         at
> org.apache.catalina.valves.RemoteAddrValve.invoke 
> (RemoteAddrValve.java:81)
>         at
> org.apache.catalina.core.StandardEngineValve.invoke 
> (StandardEngineValve.java
> :109)
>         at
> org.apache.catalina.ha.tcp.ReplicationValve.invoke 
> (ReplicationValve.java:347
> )
>         at
> org.apache.catalina.connector.CoyoteAdapter.service 
> (CoyoteAdapter.java:263)
>         at
> org.apache.coyote.http11.Http11Processor.process 
> (Http11Processor.java:844)
>         at
> org.apache.coyote.http11.Http11Protocol 
> $Http11ConnectionHandler.process(Http
> 11Protocol.java:584)
>         at
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java: 
> 447)
>         at java.lang.Thread.run(Thread.java:619)
> Dec 19, 2007 10:07:36 AM org.apache.catalina.ha.tcp.SimpleTcpCluster
> memberDisappeared
> INFO: Received member
> disappeared:org.apache.catalina.tribes.membership.MemberImpl[tcp:// 
> localhost
> :4002,localhost,4002, alive=165029291,id={-42 -48 112 17
>
> -57 -2 73 -111 -109 113 -93 84 6 91 -72 102 }, payload={}, command={},
> domain={}, ]
> Dec 19, 2007 10:07:36 AM
> org.apache.catalina.tribes.group.interceptors.TcpFailureDetector
> performBasicCheck
> INFO: Suspect member, confirmed
> dead.[org.apache.catalina.tribes.membership.MemberImpl[tcp:// 
> localhost:4002,
> localhost,4002, alive=165029291,id={-42 -48 112
>
> 17 -57 -2 73 -111 -109 113 -93 84 6 91 -72 102 }, payload={},  
> command={},
> domain={}, ]]
> Dec 19, 2007 10:07:39 AM  
> org.apache.catalina.ha.tcp.SimpleTcpCluster send
> SEVERE: Unable to send message through cluster sender.
> org.apache.catalina.tribes.ChannelException: No destination given;  
> No faulty
> members identified.
>         at
> org.apache.catalina.tribes.group.GroupChannel.send 
> (GroupChannel.java:194)
>         at
> org.apache.catalina.tribes.group.GroupChannel.send 
> (GroupChannel.java:175)
>         at
> org.apache.catalina.ha.tcp.SimpleTcpCluster.send 
> (SimpleTcpCluster.java:835)
>         at
> org.apache.catalina.ha.tcp.SimpleTcpCluster.sendClusterDomain 
> (SimpleTcpClust
> er.java:814)
>         at
> org.apache.catalina.ha.session.DeltaManager.send(DeltaManager.java: 
> 586)
>         at
> org.apache.catalina.ha.session.DeltaManager.sessionExpired 
> (DeltaManager.java
> :1211)
>         at
> org.apache.catalina.ha.session.DeltaSession.expire 
> (DeltaSession.java:366)
>         at
> org.apache.catalina.ha.session.DeltaSession.expire 
> (DeltaSession.java:352)
>         at
> org.apache.catalina.session.StandardSession.expire 
> (StandardSession.java:660)
>         at
> org.apache.catalina.session.StandardSession.invalidate 
> (StandardSession.java:
> 1111)
>         at
> org.apache.catalina.session.StandardSessionFacade.invalidate 
> (StandardSession
> Facade.java:150)
>         at pad.kernel.Resolver.service(Resolver.java:623)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java: 
> 803)
>         at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter 
> (Application
> FilterChain.java:290)
>         at
> org.apache.catalina.core.ApplicationFilterChain.doFilter 
> (ApplicationFilterCh
> ain.java:206)
>         at pad.kernel.EntryPointFilter.doFilter 
> (EntryPointFilter.java:365)
>         at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter 
> (Application
> FilterChain.java:235)
>         at
> org.apache.catalina.core.ApplicationFilterChain.doFilter 
> (ApplicationFilterCh
> ain.java:206)
>         at
> org.apache.catalina.core.StandardWrapperValve.invoke 
> (StandardWrapperValve.ja
> va:219)
>         at
> org.apache.catalina.core.StandardContextValve.invoke 
> (StandardContextValve.ja
> va:175)
>         at
> org.apache.catalina.core.StandardHostValve.invoke 
> (StandardHostValve.java:128
> )
>         at
> org.apache.catalina.valves.ErrorReportValve.invoke 
> (ErrorReportValve.java:102
> )
>         at
> org.apache.catalina.valves.RequestFilterValve.process 
> (RequestFilterValve.jav
> a:269)
>         at
> org.apache.catalina.valves.RemoteAddrValve.invoke 
> (RemoteAddrValve.java:81)
>         at
> org.apache.catalina.core.StandardEngineValve.invoke 
> (StandardEngineValve.java
> :109)
>         at
> org.apache.catalina.ha.tcp.ReplicationValve.invoke 
> (ReplicationValve.java:347
> )
>         at
> org.apache.catalina.connector.CoyoteAdapter.service 
> (CoyoteAdapter.java:263)
>         at
> org.apache.coyote.http11.Http11Processor.process 
> (Http11Processor.java:844)
>         at
> org.apache.coyote.http11.Http11Protocol 
> $Http11ConnectionHandler.process(Http
> 11Protocol.java:584)
>         at
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java: 
> 447)
>         at java.lang.Thread.run(Thread.java:619)
> Dec 19, 2007 10:07:39 AM
> org.apache.catalina.tribes.group.interceptors.TcpFailureDetector
> memberDisappeared
> INFO: Received
> memberDisappeared[org.apache.catalina.tribes.membership.MemberImpl 
> [tcp://loc
> alhost:4002,localhost,4002, alive=165033299,id={-42 -48 112 17 -57
>
> -2 73 -111 -109 113 -93 84 6 91 -72 102 }, payload={}, command={},
> domain={}, ]] message. Will verify.
> Dec 19, 2007 10:07:39 AM
> org.apache.catalina.tribes.group.interceptors.TcpFailureDetector
> memberDisappeared
> INFO: Verification complete. Member still
> alive[org.apache.catalina.tribes.membership.MemberImpl[tcp:// 
> localhost:4002,
> localhost,4002, alive=165033299,id={-42
>
> -48 112 17 -57 -2 73 -111 -109 113 -93 84 6 91 -72 102 }, payload={},
> command={}, domain={}, ]]
>
> [...]
>
> And repeats again, and again...
>
> ========================
>
>
> SERVER.XML (Instance_1)
> =======================
> [...] Connector
>
>     <Connector port="8081" protocol="HTTP/1.1"  
> maxHttpHeaderSize="8192"
> emptySessionPath="true"
>                maxThreads="150" minSpareThreads="100"  
> maxSpareThreads="300"
>                enableLookups="false" redirectPort="81443"  
> acceptCount="1000"
>                debug="0" connectionTimeout="20000"
> disableUploadTimeout="true"
>                compression="on"
> 			   compressionMinSize="2048"
> 			   noCompressionUserAgents="gozilla, traviata"
> 			   compressableMimeType="text/html,text/xml" />
>
>
> [...] Cluster Tag
>
> 	<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
>                  channelSendOptions="6">
>
>
>           <Manager  
> className="org.apache.catalina.ha.session.DeltaManager"
>                    expireSessionsOnShutdown="false"
>                    notifyListenersOnReplication="true"/>
>
>           <Channel
> className="org.apache.catalina.tribes.group.GroupChannel">
>             <Membership
> className="org.apache.catalina.tribes.membership.McastService"
>                         address="228.0.0.8"
>                         port="45578"
>                         frequency="1000"
>                         dropTime="4000"/>
>             <Receiver
> className="org.apache.catalina.tribes.transport.nio.NioReceiver"
>                       address="localhost"
>                       port="4001"
>                       autoBind="100"
>                       selectorTimeout="100"
>                       maxThreads="12"/>
>
>             <Sender
> className="org.apache.catalina.tribes.transport.ReplicationTransmitter 
> ">
>               <Transport
> className="org.apache.catalina.tribes.transport.nio.PooledParallelSend 
> er"/>
>             </Sender>
>             <Interceptor
> className="org.apache.catalina.tribes.group.interceptors.TcpFailureDet 
> ector"
> />
>             <Interceptor
> className="org.apache.catalina.tribes.group.interceptors.MessageDispat 
> ch15In
> terceptor"/>
>             <Interceptor
> className="org.apache.catalina.tribes.group.interceptors.ThroughputInt 
> ercept
> or"/>
>           </Channel>
>
>           <Valve  
> className="org.apache.catalina.ha.tcp.ReplicationValve"
>
> filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.htm;.*\.html;.*\.css;.* 
> \.txt;"/>
>
>           <Deployer
> className="org.apache.catalina.ha.deploy.FarmWarDeployer"
>                     tempDir="/tmp/war-temp/"
>                     deployDir="/tmp/war-deploy/"
>                     watchDir="/tmp/war-listen/"
>                     watchEnabled="false"/>
>           <ClusterListener
> className="org.apache.catalina.ha.session.ClusterSessionListener"/>
>         </Cluster>
> [...]
> ================
>
> WEB.XML Instance 1 and 2
> ========================
> We added the  	<Context distributable="true" /> tag
>
>
> SERVER.XML (Instance_2)
> =======================
> [...] Connector Tag
>
>     <Connector port="8082" protocol="HTTP/1.1"  
> maxHttpHeaderSize="8192"
> emptySessionPath="true"
>                maxThreads="150" minSpareThreads="100"  
> maxSpareThreads="300"
>                enableLookups="false" redirectPort="82443"  
> acceptCount="1000"
>                debug="0" connectionTimeout="20000"
> disableUploadTimeout="true"
>                compression="on"
> 			   compressionMinSize="2048"
> 			   noCompressionUserAgents="gozilla, traviata"
> 			   compressableMimeType="text/html,text/xml" />
>
> [...] Cluster Tag
>
> 	<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
>                  channelSendOptions="6">
>
>           <Manager  
> className="org.apache.catalina.ha.session.DeltaManager"
>                    expireSessionsOnShutdown="false"
>                    notifyListenersOnReplication="true"/>
>
>           <Channel
> className="org.apache.catalina.tribes.group.GroupChannel">
>             <Membership
> className="org.apache.catalina.tribes.membership.McastService"
>                         address="228.0.0.8"
>                         port="45578"
>                         frequency="1000"
>                         dropTime="4000"/>
>             <Receiver
> className="org.apache.catalina.tribes.transport.nio.NioReceiver"
>                       address="localhost"
>                       port="4002"
>                       autoBind="100"
>                       selectorTimeout="100"
>                       maxThreads="12"/>
>
>             <Sender
> className="org.apache.catalina.tribes.transport.ReplicationTransmitter 
> ">
>               <Transport
> className="org.apache.catalina.tribes.transport.nio.PooledParallelSend 
> er"/>
>             </Sender>
>             <Interceptor
> className="org.apache.catalina.tribes.group.interceptors.TcpFailureDet 
> ector"
> />
>             <Interceptor
> className="org.apache.catalina.tribes.group.interceptors.MessageDispat 
> ch15In
> terceptor"/>
>             <Interceptor
> className="org.apache.catalina.tribes.group.interceptors.ThroughputInt 
> ercept
> or"/>
>           </Channel>
>           <Valve  
> className="org.apache.catalina.ha.tcp.ReplicationValve"
>
> filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.htm;.*\.html;.*\.css;.* 
> \.txt;"/>
>           <Deployer
> className="org.apache.catalina.ha.deploy.FarmWarDeployer"
>                     tempDir="/tmp/war-temp/"
>                     deployDir="/tmp/war-deploy/"
>                     watchDir="/tmp/war-listen/"
>                     watchEnabled="false"/>
>           <ClusterListener
> className="org.apache.catalina.ha.session.ClusterSessionListener"/>
>         </Cluster>
> [...]
> ================
>
>
> Can someone help us with this problem?
> Any suggestion, idea?
>
> I can send more config files or log text if needed.
>
> Regards
> Raúl.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message