hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jaikit Savla <jai...@yahoo-inc.com>
Subject Re: AysncHttpclient TimeoutException after 2 days of running
Date Wed, 10 Apr 2013 07:20:55 GMT
Adding some more details: Total Request per second: 250  (hitting 19 hosts)

Also I am using default IOReactorConfig which has default socket timeout and connection timeout
which according to documentation is value for non blocking operation. However I do set HttpParams
in AsynClient

        HttpAsyncClient client = new DefaultHttpAsyncClient(connectionManager);
        HttpParams httpParams = client.getParams();
        HttpConnectionParams.setConnectionTimeout(httpParams, (int) config.getConnectionTimeOut().toMillis());


1. Do I also need to set sockettimeout, connectiontimeout in IOReactorConfig ?
2. Selection period is default to 1000ms => any suggestions if that is good enough for
250 rps
3.  new PoolingClientAsyncConnectionManager(ioreactor, ymailRegistry); ==> This sets timeToLive
to -1 . TimetoLive is later used to expire leased connection. Can this lead to most of the
request being timeout ?

Thanks



On Tuesday, April 9, 2013 8:53 PM, Jaikit Savla <jaikit@yahoo-inc.com> wrote:
My bad.. I pasted Ning client's initialization. Please find below Apache Httpclient initialization. 

PoolingClientAsyncConnectionManager connectionManager;
try {
    // Set up the scheme
    AsyncSchemeRegistry ymailRegistry = AsyncSchemeRegistryFactory.createDefault();
    DefaultConnectingIOReactor ioreactor = new DefaultConnectingIOReactor();
    connectionManager = new PoolingClientAsyncConnectionManager(ioreactor, ymailRegistry);
} catch (IOReactorException e) {
    throw new RuntimeException(e);
}
connectionManager.setDefaultMaxPerRoute(config.getMaxConnectionsPerService().intValue());

connectionManager.setMaxTotal(config.getMaxConnectionsTotal().intValue());

HttpAsyncClient client = new DefaultHttpAsyncClient(connectionManager);


Thanks


On Tuesday, April 9, 2013 7:06 PM, Jaikit Savla <jaikit.ydogfood@yahoo.com> wrote:
Hello, I am using Aysnc Httpclient with total of  12000 connections and 512 connection per
host. The client works fine for around 2 days and after that all the request fail with below
exception.  Any one have any clue on how to debug this ? Looks like a leak of connections
to me. Any suggestions, tips ?
    
    

        final AsyncHttpClient asyncClient = new AsyncHttpClient(
                new AsyncHttpClientConfig.Builder()
                .setCompressionEnabled(config.getCompressEnabled())
                .setConnectionTimeoutInMs(connectionTimeout)
                .setMaximumConnectionsPerHost(maxConnectionsPerService)
                .setMaximumConnectionsTotal(totalConnections).build());
        ShutdownHookSorter.INSTANCE.registerShutdownResource(
            new ShutdownResource( new Thread() {
                public void run() {
                     asyncClient.close();
                }
            }, -1.0 ) );


2013-04-10 00:35:52 Sender$HttpCallback [WARN] Failed Http request: POST http://xyz/ HTTP/1.1
: duration(MS): 50529
java.util.concurrent.TimeoutException
at org.apache.http.nio.pool.AbstractNIOConnPool.processPendingRequests(AbstractNIOConnPool.java:244)
at org.apache.http.nio.pool.AbstractNIOConnPool.lease(AbstractNIOConnPool.java:192)
at org.apache.http.impl.nio.conn.PoolingClientAsyncConnectionManager.leaseConnection(PoolingClientAsyncConnectionManager.java:176)
at org.apache.http.impl.nio.client.DefaultAsyncRequestDirector.requestConnection(DefaultAsyncRequestDirector.java:629)
at org.apache.http.impl.nio.client.DefaultAsyncRequestDirector.start(DefaultAsyncRequestDirector.java:233)
at org.apache.http.impl.nio.client.AbstractHttpAsyncClient.execute(AbstractHttpAsyncClient.java:540)
at org.apache.http.impl.nio.client.AbstractHttpAsyncClient.execute(AbstractHttpAsyncClient.java:554)
at org.apache.http.impl.nio.client.AbstractHttpAsyncClient.execute(AbstractHttpAsyncClient.java:584)
at org.apache.http.impl.nio.client.AbstractHttpAsyncClient.execute(AbstractHttpAsyncClient.java:569)
at com.yahoo.jedi.events.vespa.VespaSender.sendUpdate(VespaSender.java:59)
at com.yahoo.jedi.events.vespa.FlagEventListener.onMessage(FlagEventListener.java:101)
at org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1321)
at org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:131)
at org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:202)
at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:129)
at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:47)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)

Thanks

Jaikit

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


Mime
View raw message