jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Philippe Mouawad <philippe.moua...@gmail.com>
Subject Re: About ThreadLocal / Threads / HttpClient in JMeter
Date Sun, 14 Sep 2014 14:04:11 GMT
On Sun, Sep 14, 2014 at 3:22 PM, sebb <sebbaz@gmail.com> wrote:

> On 1 September 2014 20:25, Philippe Mouawad <philippe.mouawad@gmail.com>
> wrote:
> > Hello,
> > Working on:
> > - https://issues.apache.org/bugzilla/show_bug.cgi?id=52073
> > I am currently stucked.
> >
> > To recap the initial issue:
> > - To download embedded resources we init and shutdown a
> ThreadPoolExecutor
> > limiting the benefit of a Thread pool to the sole Sampler call.
> > - As a side effect (see below) , as this will result in the init of a new
> > HttpClient this will not perform fine due to SSLContext eager creation
> (see
> > below)
> >
> > So I started working on a patch that:
> > - Made the ThreadPoolExecutor static
> > - Used a Blocking Queue to limit the number of downloads 1 Sampler can do
> >
> > See the attached patch.
> >
> > And this is were I hit a problem.
> > Indeed we hold in ThreadLocal the HttpClient instance per Thread which
> each
> > thread needs to cleanup at end.
> > But with this design when thread ends it cleans up its state, this is a
> > problem:
> > - Currently an HttpClient will be reinitialized for each thread of the
> > Pool, this init has a cost particularly due to eager init of SSL context
> ,
> > see
> >
> http://mail-archives.apache.org/mod_mbox/hc-httpclient-users/201405.mbox/%3C1400143180.23146.7.camel@ubuntu%3E
> > - For the issue, this is a bigger problem, as I need to hold the list of
> > HttpSamplerProxy to call threadFinished at end, but this cannot work as
> the
> > list would grow huge accros the lifetime of a thread during a load test
> > ending with OOM. It grows huge because each time we call AsyncSample#call
> > we clone the HTTPSamplerBase and register it in the list. Furthermore we
> > cannot call "threadFinished" at end of Callable call (AsynSample)
> >
> >
> > Thinking about the fix I have the following ideas but they require some
> > work:
> > - First rework HC usage to only use 1 HttpClient for all threads
>
> Also need to move to latest HC version.
> I made a start on that, but found some test issues involving POST data
> formatting that I have yet to resolve.
>
> Great  !

> - Rework ThreadLocal usage:
> > * Currently we use ThreadLocal 47 times but the use is split among 2 Use
> > case:
> >  1) One is to improve performance, this one looks OK to me
> >  2) One is in fact a way to associate User Data to Thread , the thread
> > being in fact a User. I think we should rework those Thread Locals and
> move
> > them to a singleton that would be keyed on userId (could be a simple
> > increased number associated to thread). Doing so would allow us in the
> > future to move from Threads to a ThreadPool.
>
> I think we need to distinguish JMeter threads and Java threads here.
> JMeter threads correspond to Users.
>
We're on the same line

> So one way to deal with this issue would be to associate some data
> with the JMeterThread class instances.
> Either by using the class instance as the key, or by adding data to
> the class itself.
>
I think by key it would be better, maybe create a class dedicated to hold
User data.
We would put this in Thread Local when Task (of ThreadPoolExecutor) is
assigned to a Thread and remove it from Thread Local on Task end
using ThreadPoolExecutor#beforeExecute  and ThreadPoolExecutor#afterExecute


> We would need to be careful to ensure access is thread-safe as the
> data might be used by multiple Java threads.
> Also need to be careful to clean up the data when the JMeterThread
> finishes.
>
> > Thoughts ?
> >
> > Regards
> > Philippe
> > @philmdot
>



-- 
Cordialement.
Philippe Mouawad.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message