hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Oleg Kalnichevski (JIRA)" <j...@apache.org>
Subject [jira] Resolved: (HTTPCORE-118) Some strange issue under a heavy stress test
Date Sun, 30 Sep 2007 11:17:50 GMT

     [ https://issues.apache.org/jira/browse/HTTPCORE-118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Oleg Kalnichevski resolved HTTPCORE-118.
----------------------------------------

    Resolution: Invalid

Hi Halo

I do not think this is a bug in HttpCore but rather a problem with your test case. 

Consider this:
(1) In your test you open thousands of connections in a tight loop and immediately close then
after having executed just one request  
(2) You are using HTTP/1.1 protocol, but do not include 'Connection: close' header to explicitly
inform the server you do not intend to keep the connection alive.
(3) The server assumes all these thousands of connections should be kept alive. Each open
connection takes up some memory of the heap.
(4) If you start a significant number of worker threads the server eventually gets overwhelmed
with connection requests and runs out of memory

While your test case does not violate the HTTP specification, I would consider it a pretty
gross abuse of the protocol. 

Please consider re-running your tests with 'Connection: close' header included

String ReuestHead = "POST /TerminalLocationNotification HTTP/1.1\r\nHOST: LOCALHOST\r\nConnection:
close\r\nContent-Length: ";

or using another tool to stress test HttpCore that supports persistent connections such as
Apache benchmark

All stress tests I have conducted myself seem to suggest HttpCore can take a significant amount
of load

=====================================================================
With connection keep-alive option
=====================================================================
oleg@okhost:~$ ab -c 200 -n 20000 -k http://localhost:8080/index.html
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
...


Server Software:        Jakarta-HttpComponents-NIO/1.1
Server Hostname:        localhost
Server Port:            8080

Document Path:          /index.html
Document Length:        12926 bytes

Concurrency Level:      200
Time taken for tests:   8.510525 seconds
Complete requests:      20000
Failed requests:        0
Write errors:           0
Keep-Alive requests:    20000
Total transferred:      261880000 bytes
HTML transferred:       258520000 bytes
Requests per second:    2350.03 [#/sec] (mean)
Time per request:       85.105 [ms] (mean)
Time per request:       0.426 [ms] (mean, across all concurrent requests)
Transfer rate:          30050.09 [Kbytes/sec] received

=====================================================================
Without connection keep-alive option
=====================================================================
oleg@okhost:~$ ab -c 200 -n 20000 http://localhost:8080/index.html
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
...

Server Software:        Jakarta-HttpComponents-NIO/1.1
Server Hostname:        localhost
Server Port:            8080

Document Path:          /index.html
Document Length:        12926 bytes

Concurrency Level:      200
Time taken for tests:   15.643071 seconds
Complete requests:      20000
Failed requests:        0
Write errors:           0
Total transferred:      261491490 bytes
HTML transferred:       258610482 bytes
Requests per second:    1278.52 [#/sec] (mean)
Time per request:       156.431 [ms] (mean)
Time per request:       0.782 [ms] (mean, across all concurrent requests)
Transfer rate:          16324.29 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   90 624.5      0    9001
Processing:     0   56  46.6     52    1321
Waiting:        0   55  46.5     50    1321
Total:          0  147 632.1     53    9066


> Some strange issue under a heavy  stress test
> ---------------------------------------------
>
>                 Key: HTTPCORE-118
>                 URL: https://issues.apache.org/jira/browse/HTTPCORE-118
>             Project: HttpComponents Core
>          Issue Type: Bug
>          Components: Examples, HttpCore, HttpCore NIO
>    Affects Versions: 4.0-alpha5
>         Environment: Windows xp(sp2), JDK 1.5
>            Reporter: Halo chen
>            Priority: Critical
>             Fix For: 4.0-alpha5
>
>         Attachments: a.txt, SendToIA.java
>
>
> Thanks very much for your contributions.
> I use your httpcore to develop a embed web application, very well, I think. 
> But, when i do some stress test, i find a strange issue, please see my test case:
> public class SendToIA implements Runnable {
> 	String[] args;
> 	SendToIA( String[] args )
> 	{
> 		this.args = args;
> 	}
> 	public void run()
> 	{
> 		try {
> 			FileInputStream fis = new FileInputStream("a.txt");
> 			int len = fis.available();
> 			byte[] buf = new byte[len];
> 			fis.read(buf);
> 			fis.close();
> 			String ReuestHead = "POST /TerminalLocationNotification HTTP/1.1\r\nHOST: LOCALHOST\r\nContent-Length:
";
> 			String ReuestLen = String.valueOf(len) + "\r\n\r\n";
> 			String RequestBody = new String(buf);
> 			String httpRequest = ReuestHead + ReuestLen + RequestBody;			
> 			String host = "localhost";
> 			int port = 8086;
> 			if( args.length == 1 )
> 			{
> 				host = args[0];
> 			}
> 			else if( args.length == 2 )
> 			{
> 				host = args[0];
> 				port = Integer.parseInt( args[ 1 ] );
> 			}
> 			
> 			System.out.println(new java.util.Date());
> 			for (int i = 0; i < 20000; i++) {
> 				Socket connection = new Socket( host, port );
> 				connection.setReuseAddress( true );
> 				PrintWriter out = new PrintWriter(connection.getOutputStream(), true);
> 				out.print(httpRequest);
> 				out.flush();
> 				connection.close();		
> 				System.out.print("\b\rcont:"+i);
> 			}
> 			System.out.println(new java.util.Date());
> 		} catch (SecurityException e) {
> 			e.printStackTrace();
> 		} catch (Exception e) {
> 			e.printStackTrace();
> 		}
> 	}
> 	public static void main(String[] args) throws IOException {
> 		int threadcount = 1;
> 		if( args.length == 3 )
> 		{
> 			threadcount = Integer.parseInt( args[ 2 ] );
> 		}
> 		Thread[] t = new Thread[ threadcount ];
> 		for( int i = 0 ; i < threadcount ; i ++ )
> 		{
> 			new Thread( new SendToIA( args )	).start();
> 		}
> 		for( int i = 0 ; i < threadcount ; i ++ )
> 		{
> 			try{t[i].join();}catch( Exception e ){}
> 		}
> 	}
> }
> Unfortunately, the application was crashed. 
> The first , I think my application has some mistake. so I run the example( org.apache.http.examples.nio.NHttpServer)
in the HttpCore binary package. Oh,No!
> The same issue was occured! 
> Why? I found the memory usage of the application reach 170M!!, and then I add the JVM
parameter -Xmx256M.Okay, the application can running continue. but after a long time, the
memory usage of the application reach 256M and then the same thing was occured! So , I think
that the HttpCore has some memory leak.
> Can you give me some advices to resolve this issue? or I have some mistakes?
> details of error occured:
> ...
> Exception in thread "I/O reactor worker thread 2" java.lang.OutOfMemoryError: Java heap
space
> ....
> java.lang.NullPointerException
> 	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.verifyWorkers(AbstractMultiworkerIOReactor.java:136)
> 	at org.apache.http.impl.nio.reactor.DefaultListeningIOReactor.execute(DefaultListeningIOReactor.java:111)
> ....
> Exception in thread "I/O reactor worker thread 8" java.lang.OutOfMemoryError: Java heap
space
> Exception in thread "Thread-6" java.lang.OutOfMemoryError: Java heap space
> Sorry , I'm chinese , my english is bad, may be i haven't explain the issue clearly.
but I need your help.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org


Mime
View raw message