jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sebb <seb...@gmail.com>
Subject Re: About ThreadLocal / Threads / HttpClient in JMeter
Date Sun, 14 Sep 2014 14:25:16 GMT
On 14 September 2014 15:04, Philippe Mouawad <philippe.mouawad@gmail.com> wrote:
> On Sun, Sep 14, 2014 at 3:22 PM, sebb <sebbaz@gmail.com> wrote:
>
>> On 1 September 2014 20:25, Philippe Mouawad <philippe.mouawad@gmail.com>
>> wrote:
>> > Hello,
>> > Working on:
>> > - https://issues.apache.org/bugzilla/show_bug.cgi?id=52073
>> > I am currently stucked.
>> >
>> > To recap the initial issue:
>> > - To download embedded resources we init and shutdown a
>> ThreadPoolExecutor
>> > limiting the benefit of a Thread pool to the sole Sampler call.
>> > - As a side effect (see below) , as this will result in the init of a new
>> > HttpClient this will not perform fine due to SSLContext eager creation
>> (see
>> > below)
>> >
>> > So I started working on a patch that:
>> > - Made the ThreadPoolExecutor static
>> > - Used a Blocking Queue to limit the number of downloads 1 Sampler can do
>> >
>> > See the attached patch.
>> >
>> > And this is were I hit a problem.
>> > Indeed we hold in ThreadLocal the HttpClient instance per Thread which
>> each
>> > thread needs to cleanup at end.
>> > But with this design when thread ends it cleans up its state, this is a
>> > problem:
>> > - Currently an HttpClient will be reinitialized for each thread of the
>> > Pool, this init has a cost particularly due to eager init of SSL context
>> ,
>> > see
>> >
>> http://mail-archives.apache.org/mod_mbox/hc-httpclient-users/201405.mbox/%3C1400143180.23146.7.camel@ubuntu%3E
>> > - For the issue, this is a bigger problem, as I need to hold the list of
>> > HttpSamplerProxy to call threadFinished at end, but this cannot work as
>> the
>> > list would grow huge accros the lifetime of a thread during a load test
>> > ending with OOM. It grows huge because each time we call AsyncSample#call
>> > we clone the HTTPSamplerBase and register it in the list. Furthermore we
>> > cannot call "threadFinished" at end of Callable call (AsynSample)
>> >
>> >
>> > Thinking about the fix I have the following ideas but they require some
>> > work:
>> > - First rework HC usage to only use 1 HttpClient for all threads
>>
>> Also need to move to latest HC version.
>> I made a start on that, but found some test issues involving POST data
>> formatting that I have yet to resolve.
>>
>> Great  !
>
>> - Rework ThreadLocal usage:
>> > * Currently we use ThreadLocal 47 times but the use is split among 2 Use
>> > case:
>> >  1) One is to improve performance, this one looks OK to me
>> >  2) One is in fact a way to associate User Data to Thread , the thread
>> > being in fact a User. I think we should rework those Thread Locals and
>> move
>> > them to a singleton that would be keyed on userId (could be a simple
>> > increased number associated to thread). Doing so would allow us in the
>> > future to move from Threads to a ThreadPool.
>>
>> I think we need to distinguish JMeter threads and Java threads here.
>> JMeter threads correspond to Users.
>>
> We're on the same line
>
>> So one way to deal with this issue would be to associate some data
>> with the JMeterThread class instances.
>> Either by using the class instance as the key, or by adding data to
>> the class itself.
>>
> I think by key it would be better, maybe create a class dedicated to hold
> User data.
> We would put this in Thread Local when Task (of ThreadPoolExecutor) is
> assigned to a Thread and remove it from Thread Local on Task end
> using ThreadPoolExecutor#beforeExecute  and ThreadPoolExecutor#afterExecute

One cannot use standard Java ThreadLocal for data that is shared
between Java threads.

>
>> We would need to be careful to ensure access is thread-safe as the
>> data might be used by multiple Java threads.
>> Also need to be careful to clean up the data when the JMeterThread
>> finishes.
>>
>> > Thoughts ?
>> >
>> > Regards
>> > Philippe
>> > @philmdot
>>
>
>
>
> --
> Cordialement.
> Philippe Mouawad.

Mime
View raw message