hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HTTPCORE-361) Please reduce object creation in HeaderGroup.getFirstHeader()
Date Mon, 18 Jul 2016 21:02:20 GMT

    [ https://issues.apache.org/jira/browse/HTTPCORE-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15383059#comment-15383059

ASF GitHub Bot commented on HTTPCORE-361:

GitHub user benbenw opened a pull request:


    Bug 59882 : JMeter memory allocations reduction

    I've been profiling jmeter memory allocation under different conditions
    the PR linked to this bug gives substantial improvements.
    Gain really depends on the test but in the worst case (compressed content, no embedded
resources) it's about 15% less allocations (in bytes)
    When not compressed (no embedded resources) and if content-length is sent by the server
my test gives 30% reduction in bytes allocated
    As always YMMV but this PR is always a win in all my different tests.
    Reduction in memory allocated comes from
    Better usage of ByteArrayOutputStream  by
      - directly accessing the buffer instead of copying it
      - or by using the commons-lang implementation which avoid copying memory around when
extending the buffer
      - or by creating the string directly from the buffer instead of a temporary copy
    Avoid unneeded string concatenation due to missing log.isDebugEnabled()
    if possible use httpclient CharArrayBuffer to avoid memory allocations
    Do not create unneeded silent exceptions
    Do not allocate 2 empty iterators on each call to executeSamplePackage
      - LinkedList.iterator() is a stinky monkey. Also see HTTPCORE-361
    Set better default size to StringBuilder buffers
    improve JOrphanUtils#replaceAllChars : provides a fast path which does not allocate memory
    **Remark** : 
    When using ssl, jsse is a major source of unneeded allocation !!!!  (sun.security.ssl.SSLContextImpl#getDefaultCipherSuiteList(boolean))
    https://bugs.openjdk.java.net/browse/JDK-8133070 should improve the situation (I did not
test it)
    the title of the bug is misleading as it's also about an internal cache beeing cleared
on each method call.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/benbenw/jmeter memory-allocation1

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #217
commit 6019c57fd9caf03d3df22ca5e7cf3888713f193f
Author: benoit <b.wiart@ubik-ingenierie.com>
Date:   2016-07-12T07:29:15Z

    Profiling show that jmeter is doing a lot of unneeded memory allocation.
    use a specialized version of ByteArrayOutputStream which return its
    internal buffer when possible, this remove an allocation of
    byte[response size] when the content length is known and respected.
    do not create a buffer + copy when generating a md5 signature
    lazily allocate the ByteArrayOutputStream buffer which is not needed
    when it's an empty response

commit 3ae3c497e182727736540526fcdedd73713b0f57
Author: benoit <b.wiart@ubik-ingenierie.com>
Date:   2016-07-12T20:11:45Z

    do not copy byte buffer on string construction

commit f1e1d0e052bc453f1e768872f4458b636edea3d1
Author: benoit <b.wiart@ubik-ingenierie.com>
Date:   2016-07-12T20:12:42Z

    avoid stringbuilder alllocation

commit e987f4af35d9f264ff530cbe60a962ed594794ca
Author: benoit <b.wiart@ubik-ingenierie.com>
Date:   2016-07-12T20:13:49Z

    re-order test for speed

commit ff991554431d1e89f88f3b0071f5de5a580613b3
Author: benoit <b.wiart@ubik-ingenierie.com>
Date:   2016-07-12T20:36:22Z

    getResponseHeaders is allocating too much memory
    try to give a better default size to the stringbuilder
    use httpclient CharArrayBuffer to avoid memory allocations

commit 196910cba6fcef9b57081ef2f41d463b4617340f
Author: benoit <b.wiart@ubik-ingenierie.com>
Date:   2016-07-12T20:47:30Z

    in the jmx file you often find :
    <stringProp name="HTTPSampler.port"></stringProp>
    this makes getPortIfSpecified throws a "silent" exception
    this is bad as it's slow and allocates memory for the trace

commit 2d215e9e8c6e3ec4a6777e78a34348c302411891
Author: benoit <b.wiart@ubik-ingenierie.com>
Date:   2016-07-12T21:25:33Z

    do not allocate 2 empty iterators on each call to executeSamplePackage.
    It's always >5% of the memory allocated during my tests

commit 7a53bdc99631c9338660cb1284be7a19c5c39eb9
Author: benoit <b.wiart@ubik-ingenierie.com>
Date:   2016-07-13T07:43:54Z

    add a fast path which does not allocate memory when there is no
    construct the buffer with a better initial size to avoid re-allocations

commit 3e2c5921ea61765137d2db6a6acfea1d327b67c9
Author: benoit <b.wiart@ubik-ingenierie.com>
Date:   2016-07-13T07:44:38Z

    avoid unneeded access to the property map and boolean parsing

commit b59e095eaf115efbfb115b56362a54595a9684de
Author: benoit <b.wiart@ubik-ingenierie.com>
Date:   2016-07-13T08:37:33Z

    add a better default sizing for the getConnectionHeaders buffer

commit e52044a306f791ced03ed015bc827d8cc602b6ae
Author: benoit <b.wiart@ubik-ingenierie.com>
Date:   2016-07-13T08:42:29Z

    re-use pattern instead of re-creating it

commit 1c6fc75952ca5be66a448ca41c05bcff1be12266
Author: benoit <b.wiart@ubik-ingenierie.com>
Date:   2016-07-13T09:45:31Z

    improve JOrphanUtils#replaceAllChars
    provides a fast path which does not allocate memory

commit dca4f94b46e0b2a24a8f38d19d761c02c2c1b79d
Author: benoit <b.wiart@ubik-ingenierie.com>
Date:   2016-07-13T12:13:54Z

    do not allocate a temporary NullProperty if it's not needed


> Please reduce object creation in HeaderGroup.getFirstHeader()
> -------------------------------------------------------------
>                 Key: HTTPCORE-361
>                 URL: https://issues.apache.org/jira/browse/HTTPCORE-361
>             Project: HttpComponents HttpCore
>          Issue Type: Improvement
>          Components: HttpCore
>    Affects Versions: 4.3
>            Reporter: Sebastiano Vigna
>            Priority: Minor
>             Fix For: 4.3.1
>  The current implementation of getFirstHeader()
>     public Header getFirstHeader(final String name) {
>         for (final Header header : headers) {
>             if (header.getName().equalsIgnoreCase(name)) {
>                 return header;
>             }
>         }
>         return null;
>     }
> uses a Java syntax that creates an iterator object for eac hcall. We are seeing a lot
of those in heap dumps. The getLastHeader() method uses a loop and has no such problems.
> We realize that it is slightly less elegant, but it would be great if you could change
this method into
>     public Header getFirstHeader(final String name) {
>         final int size = headers.size();
>         for (int i = 0; i < size; i++) {
>             final Header header = headers.get(i);
>             if (header.getName().equalsIgnoreCase(name)) {
>                 return header;
>             }
>         }
>         return null;
> }

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org

View raw message