hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "offbynull (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HTTPCORE-368) Smarter use of buffers in SSLIOSession
Date Fri, 13 Dec 2013 23:00:07 GMT

    [ https://issues.apache.org/jira/browse/HTTPCORE-368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13848034#comment-13848034
] 

offbynull commented on HTTPCORE-368:
------------------------------------

Hi Oleg,

We based this information of off heap analysis that we did on our tool. Each buffer took around
16k. There's 4 buffers being held. We're attempting to sustain somewhere around 35k to 55k
connections.

So for 35k connections, that's 16k * 4buffers * 35,000connections. That's around 2+gigs of
heap.

Before the patch, we got OutOfMemoryExceptions around 35-37k connections. After the patch,
we were able to sustain around 55k connections (we haven't tried pushing this limit any higher).

In any event, does it make sense for buffers to exist when nothing is making use of them?
I understand that this may not be the best way to get around this problem. I've heard from
my colleague that "Netty makes use of an SslBufferPool to reduce memory consumption during
SSL communication" (his words). A pool of buffers may be a good workaround to memory constraints
as well, although this may introduce some extra complexity around threading.

> Smarter use of buffers in SSLIOSession
> --------------------------------------
>
>                 Key: HTTPCORE-368
>                 URL: https://issues.apache.org/jira/browse/HTTPCORE-368
>             Project: HttpComponents HttpCore
>          Issue Type: Improvement
>          Components: HttpCore NIO
>    Affects Versions: 4.3-beta2
>            Reporter: offbynull
>              Labels: patch
>         Attachments: SSLIOSession.patch
>
>
> We're using the async client to load test our comet infrastructure. We're frequently
running in to OutOfMemoryErrors due to the ByteBuffers allocated in SSLIOSession.
> The following is a quick-and-dirty patch to SSLIOSession that removes the buffers when
they aren't needed. Could we get this (or something similar to this) added in to httpcore-nio?



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


Mime
View raw message