hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Feist <dfe...@gmail.com>
Subject Re: Thread Contention (due to use of proxys) in HttpClient 4.3
Date Tue, 13 May 2014 09:43:06 GMT
Jaikit,

Thats exactly what I was seeing, also with Java7.

Let me do some testing though, because  just because you see
contention in profiler, doesn't mean that there is also a significant
impact.  I'll report back when I've done this.

Dan


On Mon, May 12, 2014 at 10:10 PM, Jaikit Savla <jaikit.savla@yahoo.com> wrote:
> I have seen similar issues sometimes - while running with 512 concurrent connections
using HttpClient 4.3  (Noticed all threads stuck in java.lang.reflect.Proxy.getProxyClass0
in jstack log). I found some discussion online regarding improving the performance in latest
jre but have not tried running my application with jdk8 yet.
>
> http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-April/subject.html#15854
>
> Jaikit
>
> On Monday, May 12, 2014 11:17 AM, Oleg Kalnichevski <olegk@apache.org> wrote:
>
> On Mon, 2014-05-12 at 13:21 +0100, Daniel Feist wrote:
>> >> 2) Can I expect 4.2 to scale better?
>> >
>> > I do not think so. In my tests HC 4.3 performs better than 4.2. There
>> > have also been reports
>>
>> Even with high concurrency of say 200 and high TPS, for example in a
>> http proxy scenario?
>>
>
> Yes, also with over 200 concurrent connections. However, for HTTP proxy
> scenarios one may be advised to consider using a non-blocking model.
>
>> >> 3) Is there a way of configuring 4.3.3 to not use proxys?
>> >
>> > No, there is not. The cost of dynamic proxies in modern JREs is believed
>> > to be low.
>>
>> The problem I see isn't the use of proxies, it is in the creation of
>> these proxy instances which is synchnorized on a HashMap 'cache' in
>> two places in java.lang.reflect.Proxy.getProxyClass0(ClassLoader,
>> Class<?>...).  This is visible in the profiler screenshot I shared
>> too.
>>
>> This probably isn't an issue with low concurrency, or high concurrency
>> and low TPS, but in my testing/profiling it comes up as an issue.
>>
>
> I use this micro-benchmark to benchmark HttpClient performance between
> versions as well compared to other implementations.
>
> http://wiki.apache.org/HttpComponents/HttpClient3vsHttpClient4vsHttpCore
>
> I am open to the idea of replacing dynamic proxies with hand crafted
> classes, but I would like to see more evidence that
> java.lang.reflect.Proxy is indeed a bottleneck.
>
> Oleg
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


Mime
View raw message