cxf-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lucas Pouzac (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CXF-6189) Improve memory usage of UrlUtils
Date Tue, 13 Jan 2015 15:25:35 GMT

    [ https://issues.apache.org/jira/browse/CXF-6189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14275337#comment-14275337
] 

Lucas Pouzac edited comment on CXF-6189 at 1/13/15 3:25 PM:
------------------------------------------------------------

I did some testing and some modifications. For me, I think we need to better optimize the
memory usage to be quite a bit slower.

{code:java}
/**
 * V2
 */ 
private static String urlDecode(String value, String enc, boolean isPath) {
        int length = 0;
        boolean needEncode = false;
        for (int i = 0; i < value.length(); i++) {
            if (value.charAt(i) == ESCAPE_CHAR) {
                length += 1;
                needEncode = true;
            } else if (value.charAt(i) == PLUS_CHAR) {
                needEncode = true;
            }
        }

        if (needEncode) {

            final byte[] valueBytes = StringUtils.toBytes(value, enc);
            ByteBuffer in = ByteBuffer.wrap(valueBytes);
            ByteBuffer out = ByteBuffer.allocate(in.capacity() - (2 * length)); // MODIFY
            while (in.hasRemaining()) {
                final byte b = in.get();
                if (!isPath && b == PLUS_CHAR) {
                    out.put((byte) ' ');
                } else if (b == ESCAPE_CHAR) {
                    try {
                        final int u = digit16(in.get());
                        final int l = digit16(in.get());
                        out.put((byte) ((u << 4) + l));
                    } catch (final ArrayIndexOutOfBoundsException e) {
                        throw new RuntimeException("Invalid URL encoding: ", e);
                    }
                } else {
                    out.put(b);
                }
            }
            out.flip();
            return Charset.forName(enc).decode(out).toString();
        } else {
            return value;
        }
    }
{code}

For V3, replace
{code:java}
            byte[] valueBytes = value.getBytes(enc);
            ByteBuffer in = ByteBuffer.wrap(valueBytes);
            ByteBuffer out = ByteBuffer.allocate(in.capacity());
{code}
by
{code:java}
            ByteBuffer in = Charset.forName(enc).encode(CharBuffer.wrap(value));
            ByteBuffer out = ByteBuffer.allocate(in.capacity() - (2 * length));
{code}

Result for v3
 - Faster than cxf 3.0.3
 - Slower than v1 and v2
 - But Memory usage has decreased.

{code}
o.a.c.UrlUtilsBenchmark.testDecodeAsCXF3_0_3                              thrpt       60 
219953,443 ±  4342,013   ops/s
o.a.c.UrlUtilsBenchmark.testDecodeAsCXF3_0_3:@gc.count.profiled           thrpt       60 
  1344,000 ±       NaN  counts
o.a.c.UrlUtilsBenchmark.testDecodeAsCXF3_0_3:@gc.count.total              thrpt       60 
   927,000 ±       NaN  counts
o.a.c.UrlUtilsBenchmark.testDecodeAsCXF3_0_3:@gc.time.profiled            thrpt       60 
  4917,000 ±       NaN      ms
o.a.c.UrlUtilsBenchmark.testDecodeAsCXF3_0_3:@gc.time.total               thrpt       60 
  3388,000 ±       NaN      ms
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkin                         thrpt       60 
430193,040 ± 14953,635   ops/s
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkin:@gc.count.profiled      thrpt       60 
  1383,000 ±       NaN  counts
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkin:@gc.count.total         thrpt       60 
   951,000 ±       NaN  counts
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkin:@gc.time.profiled       thrpt       60 
  5155,000 ±       NaN      ms
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkin:@gc.time.total          thrpt       60 
  3501,000 ±       NaN      ms
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV2                       thrpt       60 
425701,711 ± 11373,886   ops/s
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV2:@gc.count.profiled    thrpt       60 
  1213,000 ±       NaN  counts
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV2:@gc.count.total       thrpt       60 
   825,000 ±       NaN  counts
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV2:@gc.time.profiled     thrpt       60 
  4497,000 ±       NaN      ms
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV2:@gc.time.total        thrpt       60 
  3088,000 ±       NaN      ms
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV3                       thrpt       60 
319130,222 ±  6422,034   ops/s
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV3:@gc.count.profiled    thrpt       60 
   589,000 ±       NaN  counts
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV3:@gc.count.total       thrpt       60 
   388,000 ±       NaN  counts
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV3:@gc.time.profiled     thrpt       60 
  2085,000 ±       NaN      ms
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV3:@gc.time.total        thrpt       60 
  1454,000 ±       NaN      ms
{code}

I think we should use the v3


was (Author: lpouzac):
I did some testing and some modifications. For me, I think we need to better optimize the
memory usage to be quite a bit slower.

{code:java}
/**
 * V2
 */ 
private static String urlDecode(String value, String enc, boolean isPath) {
        int length = 0;
        boolean needEncode = false;
        for (int i = 0; i < value.length(); i++) {
            if (value.charAt(i) == ESCAPE_CHAR) {
                length += 1;
                needEncode = true;
            } else if (value.charAt(i) == PLUS_CHAR) {
                needEncode = true;
            }
        }

        if (needEncode) {

            final byte[] valueBytes = StringUtils.toBytes(value, enc);
            ByteBuffer in = ByteBuffer.wrap(valueBytes);
            ByteBuffer out = ByteBuffer.allocate(in.capacity() - (2 * length)); // MODIFY
            while (in.hasRemaining()) {
                final int b = in.get();
                if (!isPath && b == PLUS_CHAR) {
                    out.put((byte) ' ');
                } else if (b == ESCAPE_CHAR) {
                    try {
                        final int u = digit16((byte) in.get());
                        final int l = digit16((byte) in.get());
                        out.put((byte) ((u << 4) + l));
                    } catch (final ArrayIndexOutOfBoundsException e) {
                        throw new RuntimeException("Invalid URL encoding: ", e);
                    }
                } else {
                    out.put((byte) b);
                }
            }
            out.flip();
            return Charset.forName(enc).decode(out).toString();
        } else {
            return value;
        }
    }
{code}

For V3, replace
{code:java}
            byte[] valueBytes = value.getBytes(enc);
            ByteBuffer in = ByteBuffer.wrap(valueBytes);
            ByteBuffer out = ByteBuffer.allocate(in.capacity());
{code}
by
{code:java}
            ByteBuffer in = Charset.forName(enc).encode(CharBuffer.wrap(value));
            ByteBuffer out = ByteBuffer.allocate(in.capacity() - (2 * length));
{code}

Result for v3
 - Faster than cxf 3.0.3
 - Slower than v1 and v2
 - But Memory usage has decreased.

{code}
o.a.c.UrlUtilsBenchmark.testDecodeAsCXF3_0_3                              thrpt       60 
219953,443 ±  4342,013   ops/s
o.a.c.UrlUtilsBenchmark.testDecodeAsCXF3_0_3:@gc.count.profiled           thrpt       60 
  1344,000 ±       NaN  counts
o.a.c.UrlUtilsBenchmark.testDecodeAsCXF3_0_3:@gc.count.total              thrpt       60 
   927,000 ±       NaN  counts
o.a.c.UrlUtilsBenchmark.testDecodeAsCXF3_0_3:@gc.time.profiled            thrpt       60 
  4917,000 ±       NaN      ms
o.a.c.UrlUtilsBenchmark.testDecodeAsCXF3_0_3:@gc.time.total               thrpt       60 
  3388,000 ±       NaN      ms
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkin                         thrpt       60 
430193,040 ± 14953,635   ops/s
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkin:@gc.count.profiled      thrpt       60 
  1383,000 ±       NaN  counts
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkin:@gc.count.total         thrpt       60 
   951,000 ±       NaN  counts
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkin:@gc.time.profiled       thrpt       60 
  5155,000 ±       NaN      ms
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkin:@gc.time.total          thrpt       60 
  3501,000 ±       NaN      ms
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV2                       thrpt       60 
425701,711 ± 11373,886   ops/s
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV2:@gc.count.profiled    thrpt       60 
  1213,000 ±       NaN  counts
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV2:@gc.count.total       thrpt       60 
   825,000 ±       NaN  counts
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV2:@gc.time.profiled     thrpt       60 
  4497,000 ±       NaN      ms
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV2:@gc.time.total        thrpt       60 
  3088,000 ±       NaN      ms
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV3                       thrpt       60 
319130,222 ±  6422,034   ops/s
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV3:@gc.count.profiled    thrpt       60 
   589,000 ±       NaN  counts
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV3:@gc.count.total       thrpt       60 
   388,000 ±       NaN  counts
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV3:@gc.time.profiled     thrpt       60 
  2085,000 ±       NaN      ms
o.a.c.UrlUtilsBenchmark.testDecodeSergeyBeryozkinV3:@gc.time.total        thrpt       60 
  1454,000 ±       NaN      ms
{code}

I think we should use the v3

> Improve memory usage of UrlUtils
> --------------------------------
>
>                 Key: CXF-6189
>                 URL: https://issues.apache.org/jira/browse/CXF-6189
>             Project: CXF
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 3.0.3
>            Reporter: Lucas Pouzac
>              Labels: performance
>         Attachments: jmh.tar.gz, screenshot-1.png
>
>
> When I run load test, I find that the management of encoding parameters of the urls is
consumer memory.
> I do not know if it is possible to optimize this part.
> Throughput of load test : 400 query/s
> ~80% GET query with 6 parameters
> ~20% POST query with 6 parameters GET and 1 payload



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message