directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Burmester <...@plushpix.com>
Subject Re: losing packets when load testing my mina app
Date Wed, 13 Jul 2005 19:55:28 GMT
I just noticed in my ethereal traffic that it is the load testing client 
that is initiating the tcp reset packets so this may be unrelated to mina
as the client currently doesn't use mina.

The server is runing mina 0.7.3 using jdk1.5.0_04 and the os/kernel is 
redhat 2.4.21-20.ELsmp

The client is fedora 2.6.10-1.741_FC3smp

What is strange is that I don't see any tcp errors on either machine 
using netstat -i eth0 -t -c 10

Both jvms have ample memory allocated.

After running the load test for about 10 seconds I start to see occasional 
read timeouts on the client.  First I thought the mina server was not 
responding but then I found that the client is actually issuing a tcp 
reset before actually trying to send it's payload to the mina server.
As a result the mina server never actually accepts the packet and the 
client after doing a tcp retransmission a few times eventually times out.

Any suggestions?

Here is a sample of what I'm seeing on the wire:

client 10.1.36.45
mina server 206.246.214.14

No.     Time        Source                Destination           Protocol 
Info
  36334 36.252292   10.1.36.45            206.246.213.14        TCP      
41622 > 11111 [SYN] Seq=0 Ack=0 Win=5840 Len=0 MSS=1460 TSV=607287694 
TSER=0 WS=2
  36335 36.252323   206.246.213.14        10.1.36.45            TCP      
11111 > 41622 [SYN, ACK] Seq=0 Ack=1 Win=5792 Len=0 MSS=1460 TSV=262940360 
TSER=607287694 WS=0
  36336 36.252583   10.1.36.45            206.246.213.14        TCP      
41622 > 11111 [RST] Seq=1 Ack=3719067569 Win=23168 Len=0
  36337 36.252669   10.1.36.45            206.246.213.14        TCP      
41622 > 11111 [ACK] Seq=1 Ack=1 Win=5840 Len=0 TSV=607287694 
TSER=262940360
  36338 36.252681   206.246.213.14        10.1.36.45            TCP      
11111 > 41622 [RST] Seq=1 Ack=3289563097 Win=0 Len=0
  36341 36.253272   10.1.36.45            206.246.213.14        TCP      
41622 > 11111 [PSH, ACK] Seq=1 Ack=1 Win=5840 Len=131 TSV=607287695 
TSER=262940360
  36342 36.253286   206.246.213.14        10.1.36.45            TCP      
11111 > 41622 [RST] Seq=1 Ack=3289563097 Win=0 Len=0
  36683 36.458444   10.1.36.45            206.246.213.14        TCP      
[TCP Retransmission] 41622 > 11111 [PSH, ACK] Seq=1 Ack=1 Win=5840 Len=131 
TSV=607287900 TSER=262940360
  36684 36.458469   206.246.213.14        10.1.36.45            TCP      
11111 > 41622 [RST] Seq=1 Ack=3289563097 Win=0 Len=0
  37496 36.868383   10.1.36.45            206.246.213.14        TCP      
[TCP Retransmission] 41622 > 11111 [PSH, ACK] Seq=1 Ack=1 Win=5840 Len=131 
TSV=607288310 TSER=262940360
  37497 36.868405   206.246.213.14        10.1.36.45            TCP      
11111 > 41622 [RST] Seq=1 Ack=3289563097 Win=0 Len=0
  39146 37.688160   10.1.36.45            206.246.213.14        TCP      
[TCP Retransmission] 41622 > 11111 [PSH, ACK] Seq=1 Ack=1 Win=5840 Len=131 
TSV=607289130 TSER=262940360
  39147 37.688178   206.246.213.14        10.1.36.45            TCP      
11111 > 41622 [RST] Seq=1 Ack=3289563097 Win=0 Len=0
  42427 39.328107   10.1.36.45            206.246.213.14        TCP      
[TCP Retransmission] 41622 > 11111 [PSH, ACK] Seq=1 Ack=1 Win=5840 Len=131 
TSV=607290770 TSER=262940360
  42428 39.328171   206.246.213.14        10.1.36.45            TCP      
11111 > 41622 [RST] Seq=1 Ack=3289563097 Win=0 Len=0
  47930 42.607621   10.1.36.45            206.246.213.14        TCP      
[TCP Retransmission] 41622 > 11111 [PSH, ACK] Seq=1 Ack=1 Win=5840 Len=131 
TSV=607294050 TSER=262940360
  47931 42.607651   206.246.213.14        10.1.36.45            TCP      
11111 > 41622 [RST] Seq=1 Ack=3289563097 Win=0 Len=0
  51244 44.795623   10.1.36.45            206.246.213.14        TCP      
41622 > 11111 [FIN, ACK] Seq=132 Ack=1 Win=5840 Len=0 TSV=607296239 
TSER=262940360
  51245 44.795631   206.246.213.14        10.1.36.45            TCP      
11111 > 41622 [RST] Seq=1 Ack=3289563097 Win=0 Len=0
  59141 49.165337   10.1.36.45            206.246.213.14        TCP      
[TCP Retransmission] 41622 > 11111 [FIN, PSH, ACK] Seq=1 Ack=1 Win=5840 
Len=131 TSV=607300610 TSER=262940360
  59142 49.165362   206.246.213.14        10.1.36.45            TCP      
11111 > 41622 [RST] Seq=1 Ack=3289563097 Win=0 Len=0
  80975 62.283799   10.1.36.45            206.246.213.14        TCP      
[TCP Retransmission] 41622 > 11111 [FIN, PSH, ACK] Seq=1 Ack=1 Win=5840 
Len=131 TSV=607313730 TSER=262940360
  80976 62.283822   206.246.213.14        10.1.36.45            TCP      
11111 > 41622 [RST] Seq=1 Ack=3289563097 Win=0 Len=0
 102661 88.518412   10.1.36.45            206.246.213.14        TCP      
[TCP Retransmission] 41622 > 11111 [FIN, PSH, ACK] Seq=1 Ack=1 Win=5840 
Len=131 TSV=607339970 TSER=262940360
 102662 88.518428   206.246.213.14        10.1.36.45            TCP      
11111 > 41622 [RST] Seq=1 Ack=3289563097 Win=0 Len=0


Thanks,

Alex.


On Wed, 13 Jul 2005, Vinod Panicker wrote:

> During load testing, I've also experienced connection resets and
> drops, but they disappeared when I gave the JVM more memory to play
> with.  Search in the archives for this (-Xms and something else i
> forget).
> 
> Regards,
> Vinod.
> 
> On 7/13/05, Trustin Lee <trustin@gmail.com> wrote:
> > Hi Alex,
> > 
> >  
> > 2005/7/13, Alex Burmester <adb@plushpix.com>: 
> > > Hi Trustin, I have built a protocol router on top of mina
> > > and it generally runs very well.
> >  
> >   
> > Great.
> >  
> > > I have found that when I fire up about 100 threads and pound on it with my
> > > load testing client that I do get a few lost packets that never make it to
> > > the mina server process.  I did a tcpdump and found that the server
> > > machine is issuing tcp resets on the packets that are not showing up.
> > > Not sure why yet.  I assume that it's some tcp parameters I need to bump
> > > up on the linux box but I thought I would check with you to see if there
> > > are any tunable parameters in the mina framework or in the jvm that you
> > > know of that I should check.
> >  
> >   
> > Well, are you using MINA 0.7.3?  Please try to upgrade to 0.7.3 if you're
> > using an older version.  You can configure all socket parameters by calling
> > IoSession.getConfig() and downcasting the returned object into
> > SocketSessionConfig.  To configure ServerSocketChannel, you'll have to
> > upgrade to 0.9 for now. 
> >   
> > Please let me know the result after you upgrade to 0.7.3. 
> >   
> > Trustin-- 
> > what we call human nature is actually human habit
> > --
> > http://gleamynode.net/
> 


Mime
View raw message