Mailing-List: contact dev-help@jmeter.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@jmeter.apache.org
Received-SPF: pass (athena.apache.org: domain of philippe.mouawad@gmail.com
 designates 209.85.213.179 as permitted sender)
MIME-Version: 1.0
Date: Mon, 1 Sep 2014 21:25:30 +0200
Message-ID: 
 <CAH9fUpZyycaRs3KT1gFpXqhXssvKddqtcvjsT21ovao6OpKTRg@mail.gmail.com>
Subject: About ThreadLocal / Threads / HttpClient in JMeter
From: Philippe Mouawad <philippe.mouawad@gmail.com>
To: "dev@jmeter.apache.org" <dev@jmeter.apache.org>
Content-Type: multipart/alternative; boundary=90e6ba6e848849e2bc050205f7b0

--90e6ba6e848849e2bc050205f7b0
Content-Type: text/plain; charset=ISO-8859-1

Hello,
Working on:
- https://issues.apache.org/bugzilla/show_bug.cgi?id=52073
I am currently stucked.

To recap the initial issue:
- To download embedded resources we init and shutdown a ThreadPoolExecutor
limiting the benefit of a Thread pool to the sole Sampler call.
- As a side effect (see below) , as this will result in the init of a new
HttpClient this will not perform fine due to SSLContext eager creation (see
below)

So I started working on a patch that:
- Made the ThreadPoolExecutor static
- Used a Blocking Queue to limit the number of downloads 1 Sampler can do

See the attached patch.

And this is were I hit a problem.
Indeed we hold in ThreadLocal the HttpClient instance per Thread which each
thread needs to cleanup at end.
But with this design when thread ends it cleans up its state, this is a
problem:
- Currently an HttpClient will be reinitialized for each thread of the
Pool, this init has a cost particularly due to eager init of SSL context ,
see
http://mail-archives.apache.org/mod_mbox/hc-httpclient-users/201405.mbox/%3C1400143180.23146.7.camel@ubuntu%3E
- For the issue, this is a bigger problem, as I need to hold the list of
HttpSamplerProxy to call threadFinished at end, but this cannot work as the
list would grow huge accros the lifetime of a thread during a load test
ending with OOM. It grows huge because each time we call AsyncSample#call
we clone the HTTPSamplerBase and register it in the list. Furthermore we
cannot call "threadFinished" at end of Callable call (AsynSample)


Thinking about the fix I have the following ideas but they require some
work:
- First rework HC usage to only use 1 HttpClient for all threads
- Rework ThreadLocal usage:
* Currently we use ThreadLocal 47 times but the use is split among 2 Use
case:
 1) One is to improve performance, this one looks OK to me
 2) One is in fact a way to associate User Data to Thread , the thread
being in fact a User. I think we should rework those Thread Locals and move
them to a singleton that would be keyed on userId (could be a simple
increased number associated to thread). Doing so would allow us in the
future to move from Threads to a ThreadPool.

Thoughts ?

Regards
Philippe
@philmdot

--90e6ba6e848849e2bc050205f7b0--