commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Neidhart (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead
Date Tue, 17 Feb 2015 23:26:12 GMT

     [ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Thomas Neidhart updated IO-468:
-------------------------------
    Attachment: micro-benchmark.zip

I took the time to re-write my benchmark using jmh.

The results are somehow similar, but more accurate (Score is in ns/operation):

{noformat}
Benchmark                         (length)  Mode  Cnt       Score       Error  Units
IOUtilsBenchmark.copyFixedArray        100  avgt   20     255.966 ±     1.580  ns/op
IOUtilsBenchmark.copyFixedArray       1000  avgt   20     349.467 ±     3.376  ns/op
IOUtilsBenchmark.copyFixedArray      10000  avgt   20    1735.641 ±    47.352  ns/op
IOUtilsBenchmark.copyFixedArray     100000  avgt   20   20991.194 ±   253.866  ns/op
IOUtilsBenchmark.copyFixedArray    1000000  avgt   20  253168.783 ± 35859.304  ns/op
IOUtilsBenchmark.copyNormal            100  avgt   20    2039.469 ±    60.206  ns/op
IOUtilsBenchmark.copyNormal           1000  avgt   20    2210.150 ±    69.705  ns/op
IOUtilsBenchmark.copyNormal          10000  avgt   20    3467.281 ±    76.302  ns/op
IOUtilsBenchmark.copyNormal         100000  avgt   20   23033.581 ±   323.091  ns/op
IOUtilsBenchmark.copyNormal        1000000  avgt   20  251249.970 ± 23086.488  ns/op
IOUtilsBenchmark.copyThreadLocal       100  avgt   20     263.559 ±     0.985  ns/op
IOUtilsBenchmark.copyThreadLocal      1000  avgt   20     359.917 ±     4.831  ns/op
IOUtilsBenchmark.copyThreadLocal     10000  avgt   20    1795.665 ±    43.034  ns/op
IOUtilsBenchmark.copyThreadLocal    100000  avgt   20   20777.176 ±   206.086  ns/op
IOUtilsBenchmark.copyThreadLocal   1000000  avgt   20  251769.416 ± 19318.802  ns/op

{noformat}

To run the benchmark do the following on the command line:

{noformat}
  mvn clean install
  java -jar target/benchmarks.jar
{noformat}

> Avoid allocating memory for method internal buffers, use threadlocal memory instead
> -----------------------------------------------------------------------------------
>
>                 Key: IO-468
>                 URL: https://issues.apache.org/jira/browse/IO-468
>             Project: Commons IO
>          Issue Type: Improvement
>          Components: Utilities
>    Affects Versions: 2.4
>         Environment: all environments
>            Reporter: Bernd Hopp
>            Priority: Minor
>              Labels: newbie, performance
>             Fix For: 2.5
>
>         Attachments: PerfTest.java, micro-benchmark.zip, monitoring_with_threadlocals.png,
monitoring_without_threadlocals.png, performancetest.ods, performancetest_weakreference.ods
>
>   Original Estimate: 12h
>  Remaining Estimate: 12h
>
> In a lot of places, we allocate new buffers dynamically via new byte[]. This is a performance
drawback since many of these allocations could be avoided if we would use threadlocal buffers
that can be reused. For example, consider the following code from IOUtils.java, ln 2177:
> return copyLarge(input, output, inputOffset, length, new byte[DEFAULT_BUFFER_SIZE]);
> This code allocates new memory for every copy-process, that is not used outside of the
method and could easily and safely reused, as long as is is thread-local. So instead of allocating
new memory, a new utility-class could provide a thread-local bytearray like this:
> byte[] buffer = ThreadLocalByteArray.ofSize(DEFAULT_BUFFER_SIZE);
> return copyLarge(input, output, inputOffset, length, buffer);
> I have not measured the performance-benefits yet, but I would expect them to be significant,
especially when the streams itself are not the performance bottleneck. 
> Git PR is at https://github.com/apache/commons-io/pull/6/files



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message