hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Kalnichevski <ol...@apache.org>
Subject Re: scala: apache httpclient in multi-threaded environment. confusion on correct way of use
Date Sat, 12 Nov 2016 15:40:26 GMT
On Sat, 2016-11-12 at 20:16 +0530, Nishant Kumar wrote:
> Hi,
> 
> I am writing a singleton class (Object in scala) which uses apache
> httpclient(4.5.2) to send some file or some json content to a server and
> return status to caller. Here is my class -
> 
>     object HttpUtils{
>       protected val retryHandler = new HttpRequestRetryHandler() {
>         def retryRequest(exception: IOException, executionCount: Int,
> context: HttpContext): Boolean = {
>           //retry logic
>           true
>         }
>       }
>       private val connectionManager = new
> PoolingHttpClientConnectionManager()
> 
>       // Reusing same client for each request that might be coming from
> different threads .
>       // Is it correct ????

Yes.

>       val httpClient = HttpClients.custom()
>         .setConnectionManager(connectionManager)
>         .setRetryHandler(retryHandler)
>         .build()
> 
>       def restApiCall (url : String, rDD: RDD[SomeMessage]) : Boolean = {
>         // Creating new context for each request
>         val httpContext: HttpClientContext = HttpClientContext.create
>         val post = new HttpPost(url)
> 
>         // convert RDD to text file using rDD.collect
> 
>         // add this file as MultipartEntity to post
> 
>         var response = None: Option[CloseableHttpResponse] // Is it correct
> way of using it ???

Not really. HttpClient never returns null response messages. The
'response' instance is non-optional.

>         try {
>           response = Some(httpClient.execute(post, httpContext))
>           val responseCode = response.get.getStatusLine.getStatusCode
>           EntityUtils.consume(response.get.getEntity) // Is it require ???

Not required but advised if you want to ensure re-use of persistent
connections.

>           if (responseCode == 200) true
>           else false
>         }
>         finally {
>           if (response.isDefined) response.get.close
>           post.releaseConnection() // Is it require ???

No, it is not.

>         }
>       }
>       def onShutDown = {
>         connectionManager.close()
>         httpClient.close()

CloseableHttpClient#close also shuts down the connection pool associated
with it.

Oleg

>       }
>     }
> 
> 
> Multiple threads (More specifically from spark streaming context) are
> calling `restApiCall` method. I am relatively new to `scala` and `apache
> httpClient`. I have to make frequent connections to only few fixed server
> (i.e. 5-6 fixed URL's with different request parameters). I have also
> written comments in above code as question. Please address them as well.
> 
> I went through multiple online resource but still not confident about it.
> 
>  - Is it the best way to use http client in multi-threaded environment?
>  - Is it possible to keep live connections and use it for various requests
> ? Will it be beneficial in this case ?
>  - Am i using/releasing all resources efficiently ? If not please suggest.
>  - Is it good to use it in Scala or there exist some better library ?
> 
>  Thanks for your help in advance.
> 
> 



---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


Mime
View raw message