hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dinesh Babu <dinesh.b...@pb.com>
Subject RE: AbstractConnPool (httpcore-4.3.2.jar)- how to increase "available" connections
Date Fri, 23 Jan 2015 17:41:50 GMT
We found the fix. Sharing just in case if any one comes across this issue.

The root cause was because Apache Connection pool mechanism does not seem to allow more than
2 connections for a client at any given time and we needed 3 connections. So what we did was
to use a new client for each call. So the code was changed from

public ClientHttpRequest createRequest(URI uri, HttpMethod httpMethod) throws IOException
{
                if (requestFactory == null) {
                        HttpClient client = buildClient();
                        requestFactory = new HttpComponentsClientHttpRequestFactory(client);
                }

                return requestFactory.createRequest(uri, httpMethod);
        }

to

public ClientHttpRequest createRequest(URI uri, HttpMethod httpMethod) throws IOException
{
                HttpClient client = buildClient();
                requestFactory = new HttpComponentsClientHttpRequestFactory(client);
                return requestFactory.createRequest(uri, httpMethod);
}

Regards,
Dinesh Babu.

-----Original Message-----
From: Dinesh Babu [mailto:dinesh.babu@pb.com]
Sent: 22 January 2015 17:14
To: HttpClient User Discussion
Subject: RE: AbstractConnPool (httpcore-4.3.2.jar)- how to increase "available" connections

Just to pinpoint, the method used is

        public <T> T get(String resource, Class<T> responseType) {
                URI url = buildUrl(resource);
                String fullJson = this.exchange(url, HttpMethod.GET, buildRequest(""), String.class).getBody();
                return readDataElement(fullJson, responseType);
        }


Regards,
Dinesh Babu.



-----Original Message-----
From: Dinesh Babu [mailto:dinesh.babu@pb.com]
Sent: 22 January 2015 16:58
To: HttpClient User Discussion
Subject: RE: AbstractConnPool (httpcore-4.3.2.jar)- how to increase "available" connections

Thanks Stéphane,

Below is the code.


package com.pb.viewpoint.rest.client.spring;

import static org.apache.commons.lang3.StringUtils.isNotEmpty;

import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;
import java.nio.charset.Charset;
import java.security.KeyStore;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;

import com.pb.viewpoint.rest.client.ClientCustomization;
import com.pb.viewpoint.rest.client.NoClientCustomization;
import com.pb.viewpoint.rest.client.RestClient;
import com.pb.viewpoint.rest.client.RestClientException;
import org.apache.http.HttpHost;
import org.apache.http.client.HttpClient;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.SSLContexts;
import org.apache.http.conn.ssl.TrustStrategy;
import org.apache.http.impl.client.HttpClientBuilder;
import org.codehaus.jackson.JsonNode;
import org.codehaus.jackson.JsonProcessingException;
import org.codehaus.jackson.map.ObjectMapper;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpMethod;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.http.client.ClientHttpRequest;
import org.springframework.http.client.ClientHttpRequestFactory;
import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
import org.springframework.web.client.RestTemplate;

import com.pb.viewpoint.rest.client.RestClientOptions;

import javax.net.ssl.SSLContext;

public class SpringRestClient extends RestTemplate implements RestClient, ClientHttpRequestFactory
{

        private final String proxyHost;
        private final int proxyPort;
        private final ClientCustomization customization;
        private final String baseUri;
        private final String VIEWPOINT_JSON_DATA_PATH = "data";
        private String objectRootPath = VIEWPOINT_JSON_DATA_PATH;
        private HttpComponentsClientHttpRequestFactory requestFactory;
        private MediaType mediaType;
        private int timeout = 120;
        private String userAgent = "common-rest-client";
        private static final Logger LOGGER = LoggerFactory.getLogger(SpringRestClient.class);

        public SpringRestClient(RestClientOptions options) {
                this(options, new NoClientCustomization());
        }

        public SpringRestClient(RestClientOptions options, ClientCustomization customization)
{
                this.timeout = options.getTimeout();
                this.proxyHost = options.getProxyHost();
                this.proxyPort = options.getProxyPort();
                this.userAgent = options.getUserAgent();
                this.setContentType(options.getContentType());
                this.customization = customization;
                this.baseUri = options.getBaseUri();

                this.setRequestFactory(this);
                this.setErrorHandler(new CustomResponseErrorHandler());
        }

        private void setContentType(String contentType) {
                String[] splits = contentType.split("/");
                if (splits.length != 2) {
                        throw new RestClientException(String.format("ContentType '%s' is not
valid", contentType));
                }
                this.mediaType = new MediaType(splits[0], splits[1], Charset.forName("utf-8"));
        }

        public String get(URI url) {
                HttpEntity<String> entity = new HttpEntity<String>("parameters",
constructHttpHeader());
                ResponseEntity<String> responseEntity = this.exchange(url, HttpMethod.GET,
entity, String.class);
                return responseEntity.getBody();
        }

        private HttpHeaders constructHttpHeader() {
                HttpHeaders headers = new HttpHeaders();
                MediaType mediaType = new MediaType("application", "json");
                headers.setContentType(mediaType);
                return headers;
        }

        public <T> T get(String resource, Class<T> responseType) {
                URI url = buildUrl(resource);
                String fullJson = this.exchange(url, HttpMethod.GET, buildRequest(""), String.class).getBody();
                return readDataElement(fullJson, responseType);
        }

        public <T> void create(URI url, Object object, Class<T> responseType)
{
                this.postForObject(url, object, responseType);
        }

        public <T> void create(String resource, Object object, Class<T> responseType)
{
                URI url = buildUrl(resource);
                create(url, object, responseType);
        }

        public void update(String resource, Object object, String id) {
                URI url = buildUrl(resource + "/" + id);
                this.put(url, object);
        }

        public void delete(String resource, String id) {
                HttpEntity<String> entity = new HttpEntity<String>(constructHttpHeader());
                URI url = buildUrl(resource + "/" + id);
                this.exchange(url, HttpMethod.DELETE, entity, String.class);
        }

        public void delete(URI url, String body) {
                HttpEntity<String> entity = new HttpEntity(body, constructHttpHeader());
                this.postForEntity(url, entity, String.class);
        }

        private <T> HttpEntity<T> buildRequest(T entity) {
                HttpHeaders headers = new HttpHeaders();

                headers.set(org.apache.http.HttpHeaders.ACCEPT, mediaType.toString());
                headers.setContentType(mediaType);
                headers.set(org.apache.http.HttpHeaders.USER_AGENT, userAgent);

                return new HttpEntity<T>(entity, headers);
        }

        private URI buildUrl(String resource) {
                try {
                        return new URI(baseUri + "/" + resource);
                } catch (URISyntaxException e) {
                        throw new RestClientException(e.getMessage(), e);
                }
        }

        private <T> T readDataElement(String fullJson, Class<T> responseType)
{
                try {
                        ObjectMapper mapper = new ObjectMapper();
                        JsonNode jsonNode = mapper.readTree(fullJson).findPath(objectRootPath);
                        return mapper.readValue(jsonNode, responseType);
                } catch (JsonProcessingException ex) {
                        throw new RestClientException("Unable to parse json response - " +
ex.getMessage(), ex);
                } catch (IOException ex) {
                        throw new RestClientException("Unable to parse json response - " +
ex.getMessage(), ex);
                }
        }

        @Override
        public ClientHttpRequest createRequest(URI uri, HttpMethod httpMethod) throws IOException
{
                if (requestFactory == null) {
                        HttpClient client = buildClient();
                        requestFactory = new HttpComponentsClientHttpRequestFactory(client);
                }

                return requestFactory.createRequest(uri, httpMethod);
        }

        private HttpClient buildClient() {
                HttpClientBuilder builder = HttpClientBuilder.create();

                // Fiddler Proxy
                if (isNotEmpty(proxyHost) && proxyPort > 0) {
                        HttpHost proxy = new HttpHost(proxyHost, proxyPort);
                        builder.setProxy(proxy);
                }
                builder.setDefaultRequestConfig(buildRequestConfiguration());
                customization.customize(builder);
                enableSSL(builder);
                return builder.build();
        }

        private void enableSSL(HttpClientBuilder builder) {
                try {
                        SSLContext sslContext = createSSLContext();
                        builder.setSslcontext(sslContext);

                        SSLConnectionSocketFactory sslSocketFactory = createSSLSocketFactory(sslContext);
                        builder.setSSLSocketFactory(sslSocketFactory);
                } catch(Exception e) {
                        LOGGER.error("Error enabling SSL in springRestClient");
                }
        }

        private SSLContext createSSLContext() throws Exception {
                KeyStore trustStore = KeyStore.getInstance(KeyStore.getDefaultType());
                TrustStrategy allTrust = new TrustStrategy() {
                        @Override
                        public boolean isTrusted(X509Certificate[] chain, String authType)
throws CertificateException {
                                return true;
                        }
                };
                return SSLContexts.custom().useTLS().loadTrustMaterial(trustStore, allTrust).build();
        }

        private SSLConnectionSocketFactory createSSLSocketFactory(SSLContext sslContext) {
                return new SSLConnectionSocketFactory(sslContext, SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
        }

        private RequestConfig buildRequestConfiguration() {
                RequestConfig.Builder requestConfig = RequestConfig.custom();
                requestConfig = requestConfig.setConnectTimeout(this.timeout * 1000);
                requestConfig = requestConfig.setConnectionRequestTimeout(this.timeout * 1000);
                requestConfig = requestConfig.setSocketTimeout(this.timeout * 1000);
                return requestConfig.build();
        }
}




Regards,
Dinesh Babu.


-----Original Message-----
From: Stéphane Nicoll [mailto:snicoll@pivotal.io]
Sent: 22 January 2015 16:47
To: HttpClient User Discussion
Subject: Re: AbstractConnPool (httpcore-4.3.2.jar)- how to increase "available" connections

Hey,

Looks weird. Can you share your code using the RestTemplate?

S.

On Thu, Jan 22, 2015 at 5:43 PM, Dinesh Babu <dinesh.babu@pb.com> wrote:

> Hi,
>
> I am a newbie to this group
>
> We are using Spring ReST Template to make web service calls which in
> turn uses Apache Http Client  family of classes . We are seeing time
> out error from AbstractConnPool when we make calls . Details below
>
> 1) We make our first call http://abc:8080/def. For this call to
> complete we need to make another call which is below
> 2)  http://abc:8080/efg . For this call to complete we need to make
> another call which is below
> 3)  http://abc:8080/hij
>
> In AbstractConnPool the available connection is not more than 2 at any
> given point of time which is taken by my calls in (1) and (2) .  When
> call
> (3) is made since there is no available connection, this call time out.
>
> a) Is there a solution for this scenario?
> b) Is there a way to increase the number of available connections?
>
> Regards,
> Dinesh Babu.
>
> ________________________________
>
>

________________________________

B KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKCB  [  X  ܚX KK[XZ[
  Y[ ]\ \  ][  X  ܚX P˘\X K ܙ B  ܈Y][ۘ[  [X[  K[XZ[   Y[ ]\
\  Z[˘\X K ܙ B B

________________________________

B KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKCB  [  X  ܚX KK[XZ[
  Y[
]\ \  ][  X  ܚX P˘\X K ܙ B  ܈Y][ۘ[  [X[  K[XZ[
  Y[
]\ \  Z[˘\X K ܙ B B

________________________________

Mime
View raw message