hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Oleg Kalnichevski (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HTTPCORE-483) High CPU usage after enabling chunked https responses
Date Fri, 25 Aug 2017 09:44:00 GMT

     [ https://issues.apache.org/jira/browse/HTTPCORE-483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Oleg Kalnichevski resolved HTTPCORE-483.
    Resolution: Cannot Reproduce


I am closing this issue as 'Cannot Reproduce' for now. Please re-open if there are new findings
or if manage to put together a reproducer.

If you are willing to give the latest 5.0 code a go I can pledge my personal time in support
of your efforts.


> High CPU usage after enabling chunked https responses
> -----------------------------------------------------
>                 Key: HTTPCORE-483
>                 URL: https://issues.apache.org/jira/browse/HTTPCORE-483
>             Project: HttpComponents HttpCore
>          Issue Type: Bug
>          Components: HttpCore NIO
>    Affects Versions: 4.4.6
>         Environment: Centos 7
>            Reporter: Nick Cabatoff
>         Attachments: fg1-initial-unchunked.svg, fg2-initial-chunked.svg, fg3-patched-unchunked.svg,
> I wanted persistent connections with keep-alives, and I discovered that the reason I
wasn't getting them was because my responses didn't have a content-length or transfer-encoding:
chunked.  So I did body.setChunked( true ), passed that to response.setEntity(), and it worked.
 But I noticed that my app was pegged at 100% CPU in top.  I found that the app was fine when
it started, but after servicing a single HTTPS request with chunked responses, it got stuck
in this 100% cpu state.  It turns out that the reactor thread was busy looping:  despite there
being no connections open after servicing that one request, in AbstractIOReactor.execute()
the selector.select() call continued to return non-zero, and processEvent() always found a
key that was writable.  Stepping through the code I noticed that SSLIOSession.updateEventMask()
wasn't closing the session:
> {noformat}
>         // Graceful session termination
>         if (this.status == CLOSING && this.sslEngine.isOutboundDone()
>                 && (this.endOfStream || this.sslEngine.isInboundDone())) {
>             this.status = CLOSED;
>         }
> {noformat}
> The status was CLOSING and endOfStream was true, but sslEngine.isOutboundDone() returned
> I looked at httpcore 5 and found that a new check had been added just above:
> {noformat}
> diff --git a/httpcore-nio/src/main/java/org/apache/http/nio/reactor/ssl/SSLIOSession.java
> --- a/httpcore-nio/src/main/java/org/apache/http/nio/reactor/ssl/SSLIOSession.java
> +++ b/httpcore-nio/src/main/java/org/apache/http/nio/reactor/ssl/SSLIOSession.java
> @@ -371,6 +371,11 @@
>      private void updateEventMask() {
>          // Graceful session termination
> +        if (this.status == CLOSING && !this.outEncrypted.hasData()) {
> +            this.sslEngine.closeOutbound();
> +        }
> +
> +        // Graceful session termination
>          if (this.status == CLOSING && this.sslEngine.isOutboundDone()
>                  && (this.endOfStream || this.sslEngine.isInboundDone())) {
>              this.status = CLOSED;
> {noformat}
> I applied this patch to my httpcore 4.4.6 and the problem went away.  Now after servicing
a request with chunked responses the session gets closed properly and the reactor is properly
> There's still another problem outstanding though.  Request processing now consumes way
more CPU than it used to.  Doing one request per second without chunked responses, my app
consumes ~20% of one core; with chunked responses it's closer to 100%.  
> I'm attaching flame graphs (see [http://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html]):
(1) my starting point without chunked respones; (2) with chunked respones; (3) after applying
the above patch, without chunked responses; (4) after applying patch, with chunked responses.
 Note the similarity between (2) and (4): in both cases, much of the Java code (green) is
mostly consumed with epoll calls.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org

View raw message