commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhaohua Meng" <zm...@cgsh.com>
Subject HttpClient: can it cause "too many open files" ?
Date Wed, 02 Nov 2005 23:03:03 GMT
HttpClient gugus,


I have a HttpClientEngine (see code listing at the bottom ) which wraps 
commons HttpClient. It is used as following in my code deployed on SuSe 
Linux.

for (int i=0; i<requests.length; i++)  }
        HttpClientEngine engine = new HttpClientEngine ();
        response = engine.doPost(requests[i]);
        //code to process response
}

In real situation t he request loop can be as big as 500+. After the code 
running for a while I got "too many open files" exception. I know in Linux 
a stream is considered a file. Please note that 
"method.releaseConnection()" is called before doPost() returns a response 
object. My questions are following:

1. Will the following code work? In other words, can HttpClient object be 
reused to post with different HttpBaseMethod?

HttpClientEngine engine = new HttpClientEngine ();
for (int i=0; i<requests.length; i++)  }
        response = engine.doPost(requests[i]);
        //code to process response
}

2. Will my code leave some streams open thus causes "too many open files"? 


3. Besides "method.releaseConnection()", what other methods should I call 
to close all streams?

Thanks,
Zhaohua


public class HttpClientEngine {
        private HttpClient httpClient; //this is the commons HttpClient 
        private HttpResponse response; //my proprietary class
        private HttpRequest request; //my proprietary class
        private URL url;
        private Credentials credentials;
        public HttpClientEngine() {
                httpClient = new HttpClient();
        }
        public HttpResponse doPost(HttpRequest request) {
                return 
doPost(request,request.getURL(),request.getCredentials());
        }
        public HttpResponse doPost(HttpRequest request, URL url) {
                return doPost(request,url,request.getCredentials());
        }
        private static Logger logger = Logger.getInstance();
        public HttpResponse doPost(HttpRequest request, URL url, 
Credentials credentials) {
                this.request = request;
                this.url = url;
                this.credentials = credentials;
                return service(request,new PostMethod(url.toString()));
        }

        public HttpResponse doGet(HttpRequest request) {
                return 
doGet(request,request.getURL(),request.getCredentials());
        }
        public HttpResponse doGet(HttpRequest request, URL url) {
                return doGet(request,url,request.getCredentials());
        }
        public HttpResponse doGet(HttpRequest request, URL url, 
Credentials credentials) {
                this.request = request;
                this.url = url;
                this.credentials = credentials;
                return service(request,new GetMethod(url.toString()));
        }
 
 
        private HttpResponse service(HttpRequest request, HttpMethodBase 
method) {
                HttpHeader[] requestHeaders = request.getHeaders();
                if ( requestHeaders != null) {
                        for (int i=0; i<requestHeaders.length; i++) {
                                //debug("header: 
["+requestHeaders[i].getName()+"]["+requestHeaders[i].getValue()+"]");
 
method.setRequestHeader(requestHeaders[i].getName(),requestHeaders[i].getValue());
                        }
                }
                if (credentials != null) {
                        UsernamePasswordCredentials login = 
                                new 
UsernamePasswordCredentials(credentials.getName(),credentials.getPassword()); 
 
                        AuthScope auth=  new AuthScope(url.getHost(), 
url.getPort(), null);
                httpClient.getState().setCredentials(auth, login);
                method.setDoAuthentication(true);
                }
        if (method instanceof PostMethod) {
                try {
                                ((PostMethod)method).setRequestEntity(new 
ByteArrayRequestEntity(request.getBody()));
                        } catch (Exception e) {
                                method.releaseConnection();
                                return null;
                        }
        }
        try {
                        int statusCode = httpClient.executeMethod(method);
                        Header[] responseHeaders = 
method.getResponseHeaders();
                        HttpHeader[] headers = null;
                        if (responseHeaders != null) {
                                headers = new 
HttpHeader[responseHeaders.length];
                                for (int i=0; i<responseHeaders.length; 
i++) {
                                        headers[i] = new HttpHeader();
 headers[i].setName(responseHeaders[i].getName());
 headers[i].setValue(responseHeaders[i].getValue());
                                }
                        }
 
                        response = 
handler.process(method.getResponseBody(),statusCode, headers);
                } catch (HttpException e) {
                        throw new RuntimeException(e.getMessage());
                } catch (IOException e) {
                        throw new RuntimeException(e.getMessage());
                } finally {
                        method.releaseConnection();
        }
                return response;
        }
        private HttpResponseHandler handler;
        public void setHttpResponseHandler(HttpResponseHandler handler) {
                this.handler = handler;
        }
        public static void debug(Object obj) {
 System.out.println(HttpClientEngine.class.getName()+"::"+obj.toString());
        }

Zhaohua Meng
WebSphere Consultant
This message is being sent from a law firm and may contain confidential or privileged information.
 If you are not the intended recipient, please advise the sender immediately by reply e-mail
and delete this message and any attachments without retaining a copy.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message