hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "H. Brunsting" <h...@zxcvbnm.nl>
Subject Trying to get a safe multi-threaded implementation, getting I/O exception (java.net.SocketException) caught when processing request: Connection reset
Date Tue, 14 Dec 2010 11:32:00 GMT
Hi,

I'm trying to get a multi-threaded implementation of HttpClient.
Basically i have a number of threads that call getHttpData()
concurrently. It seems to work, but upon starting my threads I get a
bunch of exceptions:

Dec 14, 2010 11:32:17 AM
org.apache.http.impl.client.DefaultRequestDirector execute
INFO: I/O exception (java.net.SocketException) caught when processing
request: Connection reset
Dec 14, 2010 11:32:17 AM
org.apache.http.impl.client.DefaultRequestDirector execute
INFO: Retrying request

After this the threads seem to actually retry and succeed in
establishing a connection, but I'd like to find out what's going wrong
here.
It doesn't happen on every run, that's what makes this difficult to
track down. The program is a server, sometimes I can make dozens of
calls without hitting that exception, then suddenly it will display
errors for a large number of requests -- all on one server instance.


My program works like this:

1 call has about 40 sources.

 for(String source : sourceList){
	SearchThread searchThread = new SearchThread(name, source, id++);
	Thread thread = new Thread(searchThread);
	thread.start();
}

SearchThread is a simple Runnable that does data =
HttpClientWrapper.getHttpData(url); in its run() body.

Can anyone shed a light on this? Thanks in advance.

Below is the full code for HttpClientWrapper:


=============================


import java.io.BufferedReader;

public class HttpClientWrapper {

        private static HttpClient httpClient;

        private static HttpClient createHttpClient(){
                HttpParams params = new BasicHttpParams();

                HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
                HttpProtocolParams.setContentCharset(params,
HTTP.DEFAULT_CONTENT_CHARSET);
                HttpProtocolParams.setUseExpectContinue(params, true);
                HttpConnectionParams.setStaleCheckingEnabled(params, false);
                HttpClientParams.setCookiePolicy(params, CookiePolicy.NETSCAPE);

                HttpConnectionParams.setConnectionTimeout(params, 15 * 1000);
                HttpConnectionParams.setSoTimeout(params, 15 * 1000);
                HttpConnectionParams.setSocketBufferSize(params, 8192);


params.setParameter(ConnManagerPNames.MAX_TOTAL_CONNECTIONS, 100);

params.setParameter(ConnManagerPNames.MAX_CONNECTIONS_PER_ROUTE, new
ConnPerRouteBean(100));

                SchemeRegistry schReg = new SchemeRegistry();
                schReg.register(new Scheme("http",
PlainSocketFactory.getSocketFactory(), 80));
                schReg.register(new Scheme("https",
SSLSocketFactory.getSocketFactory(), 443));

                ClientConnectionManager conMgr = new
ThreadSafeClientConnManager(params, schReg);

                return new DefaultHttpClient(conMgr, params);

        }

        public static HttpClient getHttpClient(){
                if(httpClient == null){
                        httpClient = createHttpClient();
                }
                return httpClient;
        }

        public void shutdown(){
                if(httpClient != null &&
httpClient.getConnectionManager() !=null){
                        httpClient.getConnectionManager().shutdown();
                }
        }

        public static String getHttpData(String url){
                HttpGet request;
                try {
                        HttpClient httpClient = getHttpClient();
                        request = new HttpGet();

                        request.setURI(new URI(url));

                        HttpResponse response = httpClient.execute(request);
                        InputStream input = response.getEntity().getContent();

                        BufferedReader reader = new BufferedReader(new
InputStreamReader(input), 8192);
                        StringBuilder sb = new StringBuilder();
                        String line = null;

                        while ((line = reader.readLine()) != null) {
                                sb.append(line + "\n");
                        }
                        request.abort();
                        return sb.toString();
                } catch (Exception e){
                        e.printStackTrace();
                }

                return null;
        }


}

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


Mime
View raw message