commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bernd Hopp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead
Date Mon, 09 Feb 2015 15:07:34 GMT

    [ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14312325#comment-14312325
] 

Bernd Hopp commented on IO-468:
-------------------------------

Thomas, I DID read the material. I responded to every point made and I repeatedly asked you
for test data showing that my implementation is in any possible test case inferior to the
current implementation, yet you did not respond.

Subclassing ThreadLocal obviously leads to problems with the webapp classloader, as Mark pointed
out, that is a valid point. But it is very easy to fix, I will change the implementation to
not subclass ThreadLocal.

As for the ~ns of allocation time: It was not 2, it was 164228458 in 4.096 runs, an average
of 40094 in one run, which is 0,04 ms. I give you that this is no big loot, but it sums up
on systems that do a lot of copying and the memory profile benefits from the patch also. 

> Avoid allocating memory for method internal buffers, use threadlocal memory instead
> -----------------------------------------------------------------------------------
>
>                 Key: IO-468
>                 URL: https://issues.apache.org/jira/browse/IO-468
>             Project: Commons IO
>          Issue Type: Improvement
>          Components: Utilities
>    Affects Versions: 2.4
>         Environment: all environments
>            Reporter: Bernd Hopp
>            Priority: Minor
>              Labels: newbie, performance
>             Fix For: 2.5
>
>         Attachments: PerfTest.java, monitoring_with_threadlocals.png, monitoring_without_threadlocals.png,
performancetest.ods, performancetest_weakreference.ods
>
>   Original Estimate: 12h
>  Remaining Estimate: 12h
>
> In a lot of places, we allocate new buffers dynamically via new byte[]. This is a performance
drawback since many of these allocations could be avoided if we would use threadlocal buffers
that can be reused. For example, consider the following code from IOUtils.java, ln 2177:
> return copyLarge(input, output, inputOffset, length, new byte[DEFAULT_BUFFER_SIZE]);
> This code allocates new memory for every copy-process, that is not used outside of the
method and could easily and safely reused, as long as is is thread-local. So instead of allocating
new memory, a new utility-class could provide a thread-local bytearray like this:
> byte[] buffer = ThreadLocalByteArray.ofSize(DEFAULT_BUFFER_SIZE);
> return copyLarge(input, output, inputOffset, length, buffer);
> I have not measured the performance-benefits yet, but I would expect them to be significant,
especially when the streams itself are not the performance bottleneck. 
> Git PR is at https://github.com/apache/commons-io/pull/6/files



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message