hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Speirs <bill.spe...@gmail.com>
Subject Re: HttpClient Performance Issues
Date Tue, 19 Jul 2011 13:41:03 GMT
I dug deeper into the performance issues I was having... they were all
disk related.

Attached is a new graph of 1,000 requests, 100 at a time through my
code. It plots having logging enabled and disabled, and registering
and not registering the https scheme. In summary, logging (which I
forgot was also going to disk) turned out to be the biggest culprit.
As the filesystem is on NFS, writing to disk was slowing everything
down quite a bit (as can be seen in red). However, what I found
interesting was the impact of adding the https scheme to the client.
This was causing great pain for my initial connections as all of the
SSL certificate stores needed to be read and parsed in preparation for
a possible SSL handshake/connection. This can be seen in green; the
high numbers in the start.

With both logging and SSL disabled, I'm able to see ~10K requests per second.

I'm putting this out there as it is hopefully a help to anyone else
having performance issues... check what you're reading/writing to


On Fri, Jul 15, 2011 at 4:59 PM, Bill Speirs <bill.speirs@gmail.com> wrote:
> Also, being network related would explain why it doesn't show up in the
> benchmark tests... they all connect to local host.
> Again though, if you could prime the client in some way to prevent this
> initial delay that could be a nice feature to have in the Apache client over
> the other clients out there: JRE, Jetty, etc.
> Bill-
> On Jul 15, 2011 4:56 PM, "Bill Speirs" <bill.speirs@gmail.com> wrote:
>> I swithced my code over to the Jetty client and did the same graphing.
>> Like
>> the Apache client I see up to 3 seconds at the start and the shoots down
>> to
>> ~20ms after. So I am inclinded to agree; I think it is DNS or some other
>> network related issue.
>> I will try and track down exactly what. Whatever it is, maybe it could be
>> "precomputed" to save time.
>> Bill-
>> On Jul 15, 2011 4:07 PM, "Oleg Kalnichevski" <olegk@apache.org> wrote:
>>> On Fri, 2011-07-15 at 15:40 -0400, Bill Speirs wrote:
>>>> On Fri, Jul 15, 2011 at 2:56 PM, Oleg Kalnichevski <olegk@apache.org>
>> wrote:
>>>> > What makes you think the time was wasted inside HttpClient code?
>>>> I've re-run the test quite a few times (both with ab and the benchmark
>>>> code) and EVERY time it's the first 100 requests. I'm only tracking
>>>> the time around the execute() function, so it's coming from something
>>>> the client is doing.
>>> Still, it may be a problem with your DNS or some other networking issue
>>> completely outside of HttpClient's control.
>>>> I don't think it's that easy to say the first ones are simply
>>>> irrelevant. In my benchmark they're all irrelevant; however, in
>>>> production there will possibly be a human sitting behind that request
>>>> (or a 100 in this case) and they will have to wait for 6 seconds
>>>> whereas everyone else will only have to wait a ~20ms.
>>> I have a hard time believing that a bug causing performance degradation
>>> from 20 milliseconds to 6 second could have gone undetected for so long,
>>> but I have mistaken before. By all of means please do continue digging.
>>> Oleg
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
>>> For additional commands, e-mail: dev-help@hc.apache.org

View raw message