Mailing-List: contact httpclient-users-help@hc.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "HttpClient User Discussion" <httpclient-users@hc.apache.org>
MIME-Version: 1.0
From: Nishant Kumar <nishantkumar35@gmail.com>
Date: Sat, 12 Nov 2016 20:16:45 +0530
Message-ID: <CAMDUb2C2hCEO3VPL+i+HrZDOcqEsd9EtpQcLa9OsnTqk1UU6=g@mail.gmail.com>
Subject: scala: apache httpclient in multi-threaded environment. confusion on
 correct way of use
To: httpclient-users@hc.apache.org
Content-Type: multipart/alternative; boundary=001a114fa61602e88705411bade1
archived-at: Sat, 12 Nov 2016 14:46:53 -0000

--001a114fa61602e88705411bade1
Content-Type: text/plain; charset=UTF-8

Hi,

I am writing a singleton class (Object in scala) which uses apache
httpclient(4.5.2) to send some file or some json content to a server and
return status to caller. Here is my class -

    object HttpUtils{
      protected val retryHandler = new HttpRequestRetryHandler() {
        def retryRequest(exception: IOException, executionCount: Int,
context: HttpContext): Boolean = {
          //retry logic
          true
        }
      }
      private val connectionManager = new
PoolingHttpClientConnectionManager()

      // Reusing same client for each request that might be coming from
different threads .
      // Is it correct ????
      val httpClient = HttpClients.custom()
        .setConnectionManager(connectionManager)
        .setRetryHandler(retryHandler)
        .build()

      def restApiCall (url : String, rDD: RDD[SomeMessage]) : Boolean = {
        // Creating new context for each request
        val httpContext: HttpClientContext = HttpClientContext.create
        val post = new HttpPost(url)

        // convert RDD to text file using rDD.collect

        // add this file as MultipartEntity to post

        var response = None: Option[CloseableHttpResponse] // Is it correct
way of using it ???
        try {
          response = Some(httpClient.execute(post, httpContext))
          val responseCode = response.get.getStatusLine.getStatusCode
          EntityUtils.consume(response.get.getEntity) // Is it require ???
          if (responseCode == 200) true
          else false
        }
        finally {
          if (response.isDefined) response.get.close
          post.releaseConnection() // Is it require ???
        }
      }
      def onShutDown = {
        connectionManager.close()
        httpClient.close()
      }
    }


Multiple threads (More specifically from spark streaming context) are
calling `restApiCall` method. I am relatively new to `scala` and `apache
httpClient`. I have to make frequent connections to only few fixed server
(i.e. 5-6 fixed URL's with different request parameters). I have also
written comments in above code as question. Please address them as well.

I went through multiple online resource but still not confident about it.

 - Is it the best way to use http client in multi-threaded environment?
 - Is it possible to keep live connections and use it for various requests
? Will it be beneficial in this case ?
 - Am i using/releasing all resources efficiently ? If not please suggest.
 - Is it good to use it in Scala or there exist some better library ?

 Thanks for your help in advance.


-- 
Nishant Kumar
Bangalore, India
Mob: +91 80088 42030
Email: nishantkumar35@gmail.com

--001a114fa61602e88705411bade1--