hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ortwin Glück <ortwin.glu...@nose.ch>
Subject Re: Performance HTTPClient vs. ab
Date Thu, 14 Aug 2003 07:05:31 GMT
Wolfgang,

please try using the SimpleHttpConnectionManager instead of 
MultiThreadedHttpConnectionManager.

Odi

Wolfgang Hoschek wrote:
> I am planning on sending and receiving MANY (tens of tousands) small 
> SOAP-like serial messages over the same persistent HTTP 1.1 connection, 
> using a single client and thread (nothing fancy there). A High 
> performance, low latency http library would be ideal.
> 
> HTTPClient performance seems devastating wrt. ab (apache benchmark): 
> 4300 requests/sec vs. 20 requests/sec, i.e. two orders of magnitute 
> difference, no matter what options I try with HTTPClient.
> 
> Source code to reproduce and full data is attached.
> If anyone is interested, please let me know what I'm doing wrong, 
> reproduce the effect and/or post working code that performs well in your 
> experience.
> 
> ab -k -n 10000 http://localhost:8080/discofish/HelloWorldServlet > /tmp/out
> 
> Requests per second:    4313.48 [#/sec] (mean)
> 
> (For more data see full output in attachment)
> 
> 
> [doggy /home/oliver/g5/users/hoschek] time disco-java.sh 
> gov.lbl.dsd.discofish.client.HTTPBug 100 
> http://localhost:8080/discofish/HelloWorldServlet > /tmp/out2
> 
> real    0m4.876s
> user    0m0.550s
> sys     0m0.060s
> 
> --> Requests per second:    20.0
> 
> (For more data see full output in attachment)
> 
> Similar performance gap on static HTML pages (no need to use a servlet).
> 
> Environment: httpclient-2.0rc1, jdk-1.4.2, tomcat-4.1.27 (server.xml out 
> of the box), 2GHz Pentium IV, Redhat 8.0.
> 
> Thanks,
> Wolfgang.
> 
> 
> ------------------------------------------------------------------------
> 
> This is ApacheBench, Version 2.0.40-dev <$Revision: 1.116 $> apache-2.0
> Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
> Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/
> 
> Benchmarking localhost (be patient)
> 
> 
> Server Software:        Apache
> Server Hostname:        localhost
> Server Port:            8080
> 
> Document Path:          /discofish/HelloWorldServlet
> Document Length:        8 bytes
> 
> Concurrency Level:      1
> Time taken for tests:   2.210253 seconds
> Complete requests:      10000
> Failed requests:        0
> Write errors:           0
> Keep-Alive requests:    9900
> Total transferred:      1579500 bytes
> HTML transferred:       80000 bytes
> Requests per second:    4524.37 [#/sec] (mean)
> Time per request:       0.221 [ms] (mean)
> Time per request:       0.221 [ms] (mean, across all concurrent requests)
> Transfer rate:          697.66 [Kbytes/sec] received
> 
> Connection Times (ms)
>               min  mean[+/-sd] median   max
> Connect:        0    0   0.0      0       0
> Processing:     0   11   7.6     11      37
> Waiting:        0    0   0.5      0      17
> Total:          0   11   7.6     11      37
> 
> Percentage of the requests served within a certain time (ms)
>   50%     11
>   66%     14
>   75%     16
>   80%     17
>   90%     19
>   95%     26
>   98%     32
>   99%     35
>  100%     37 (longest request)
> 
> 
> ------------------------------------------------------------------------
> 
> 
> ......................................................................
> ..............................goodbye
> 
> 
> ------------------------------------------------------------------------
> 
> package gov.lbl.dsd.discofish.client;
> 
> import java.io.IOException;
> 
> import org.apache.commons.httpclient.Header;
> import org.apache.commons.httpclient.HttpClient;
> import org.apache.commons.httpclient.HttpRecoverableException;
> import org.apache.commons.httpclient.MultiThreadedHttpConnectionManager;
> import org.apache.commons.httpclient.methods.PostMethod;
> 
> 
> public class HTTPBug {
> 	
> 	protected HttpClient client;
> 	protected String url;
> 	
> 	public static void main(String[] args) throws Exception {
> 		int k=0;
> 
> 		int size = 1000;
> 		if (args.length > k) size = Integer.parseInt(args[k++]);
> 		
> 		//String url = "http://localhost:8080/discofish/HelloWorldServlet";
> 		String url = "http://localhost:8080/discofish/index.html";
> 		//String url = "http://itg.lbl.gov/~hoschek/discofish/disco-grep-usage.txt";
> 		if (args.length > k) url = args[k++];
> 		
> 		MultiThreadedHttpConnectionManager connMgr = new MultiThreadedHttpConnectionManager();
> 		//connMgr.setMaxConnectionsPerHost(200);
> 		//connMgr.setMaxTotalConnections(200);
> 		HttpClient client = new HttpClient(connMgr);
> 		//HttpClient client = new HttpClient();
> 		
> 		for (int i=0; i<size; i++) {
> 			PostMethod method = new PostMethod(url);
> 			method.setRequestHeader("Content-type", "text/xml; charset=ISO-8859-1");
> 			method.setRequestHeader("Accept", "application/soap+xml, application/dime, multipart/related,
text/*");
> 			method.setRequestHeader("SOAPAction", "");
> 			//method.setRequestContentLength(PostMethod.CONTENT_LENGTH_CHUNKED);			
> 			//method.setRequestBody(XMLUtil.msg2SoapString(new Receive(new TransactionID(), 1,
1, System.currentTimeMillis() + 10000, "single")));
> 			method.setRequestBody("hello");
> 			//method.setRequestContentLength(5);
> 			int statusCode = -1;
> 
> 			/*Header[] headers;
> 			headers = method.getRequestHeaders();
> 			for (int j=0; j < headers.length; j++) {
> 				System.out.print(headers[j].toString());
> 			}
> 
> 			String requestBody = method.getRequestBodyAsString();
> 			System.out.println(requestBody);
> 			System.out.println();*/
> 
> 			
> 			try {
> 				// execute the method.
> 				statusCode = client.executeMethod(method);
> 			} catch (HttpRecoverableException e) {
> 				System.out.println("A recoverable exception occurred, retrying.  " + e.getMessage());
> 			} catch (IOException e) {
> 				System.out.println("Failed to download file.");
> 				e.printStackTrace();
> 				System.exit(-1);
> 			}
> 			// Check that we didn't run out of retries.
> 			if (statusCode == -1) {
> 				System.out.println("Failed to recover from exception.");
> 				System.exit(-2);
> 			}
> 			
> 			//String responseBody = method.getResponseBodyAsString();
> 
> 			/*headers = method.getResponseHeaders();
> 			for (int j=0; j < headers.length; j++) {
> 				System.out.print(headers[j].toString());
> 			}
> 
> 			System.out.println(responseBody);*/
> 			if (i % 70 == 0) System.out.println();
> 			System.out.print('.');
> 
> 			method.releaseConnection();
> 			//method.recycle();
> 		}
> 		
> 		System.out.println("goodbye");
> 	}
> 
> }
> 
> 
> ------------------------------------------------------------------------
> 
> package gov.lbl.dsd.discofish.server;
> 
> import java.io.ByteArrayOutputStream;
> import java.io.IOException;
> import java.io.PrintWriter;
> 
> import javax.servlet.ServletException;
> import javax.servlet.http.HttpServlet;
> import javax.servlet.http.HttpServletRequest;
> import javax.servlet.http.HttpServletResponse;
> 
> public class HelloWorldServlet extends HttpServlet {
> 
> 	public void doPost(HttpServletRequest request, HttpServletResponse response) throws
IOException, ServletException {
> 		doGet(request, response);
> 	}
> 	
> 	public void doGet(HttpServletRequest request, HttpServletResponse response)
> 		throws IOException, ServletException {
> 		
> 		response.setContentType("text/xml");
> 
> 		ByteArrayOutputStream bytes = new ByteArrayOutputStream(1000);
> 		PrintWriter out = new PrintWriter(bytes, true);  // true forces flushing
> 
> 		out.println("</HTML>");
> 
> 		// Set the content length to the size of the buffer
> 		response.setContentLength(bytes.size());
> 
> 		// Send the buffer
> 		bytes.writeTo(response.getOutputStream());
> 			
> 		/*out.flush();
> 		out.close();*/
> 		//response.flushBuffer();
> 	}
> 
> }
> 
> 
> 
> ------------------------------------------------------------------------
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org

-- 
  _________________________________________________________________
  NOSE applied intelligence ag

  ortwin glück                      [www]      http://www.nose.ch
  software engineer                 [email] ortwin.glueck@nose.ch
  hardturmstrasse 171               [pgp key]          0x81CF3416
  8005 zurich                       [office]      +41-1-277 57 35
  switzerland                       [fax]         +41-1-277 57 12


Mime
View raw message